2024-04-02T15:57:15.416 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T15:57:15.416 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686 --description fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/orig.config.yaml 2024-04-02T15:57:15.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T15:57:15.555 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686 branch: wip-batrick-testing-20240402.004512 description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} email: pdonnell@redhat.com first_in_suite: false job_id: '7635686' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRkQdXxPy/y2GsjXPcZmsdMRTTfTw3lb43j9ELZzcr0kLRrXHgO+ltSrf0OT/QX3M+AIKGCfHZzUFk8gB5NL9g= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPVksMOgDS4ZluSQ96fhgGWRK7u4t7F6s4evIu+rAK70Q3Q9d12fWJFVIIu/bU7DyogNnL260EZII3oxw/t1W8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T15:57:15.556 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T15:57:15.556 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T15:57:15.556 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T15:57:15.558 INFO:teuthology.task.internal:Checking packages... 2024-04-02T15:57:15.580 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T15:57:15.580 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T15:57:15.580 INFO:teuthology.packaging:ref: None 2024-04-02T15:57:15.580 INFO:teuthology.packaging:tag: None 2024-04-02T15:57:15.580 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T15:57:15.580 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T15:57:15.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T15:57:15.793 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609-g191d963a-1jammy 2024-04-02T15:57:15.794 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T15:57:15.801 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T15:57:15.801 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T15:57:15.819 INFO:teuthology.task.internal:Saving configuration 2024-04-02T15:57:15.835 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T15:57:15.841 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T15:57:15.867 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 15:51:04.939783', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHHNyM8LDtl9fBLruwoytXmXg0NZB7qJm2FSDJNLZnq7'} 2024-04-02T15:57:15.891 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 15:51:04.941374', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC6mkybElAqQVDgrOzt+D5Wz3hRUQAbl4z4/e3Hwjelt'} 2024-04-02T15:57:15.892 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T15:57:15.898 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-02T15:57:15.898 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-02T15:57:15.898 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T15:57:15.968 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa4073633a0>, signals=[15]) 2024-04-02T15:57:15.969 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T15:57:15.975 INFO:teuthology.task.internal:Opening connections... 2024-04-02T15:57:15.975 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T15:57:15.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:57:16.058 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T15:57:16.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:57:16.133 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T15:57:16.140 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-02T15:57:16.145 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-02T15:57:16.145 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="22.04" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=jammy 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-02T15:57:16.194 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=jammy 2024-04-02T15:57:16.195 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-02T15:57:16.213 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-04-02T15:57:16.218 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-04-02T15:57:16.219 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="22.04" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=jammy 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-02T15:57:16.266 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-02T15:57:16.267 INFO:teuthology.orchestra.run.smithi204.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-02T15:57:16.267 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=jammy 2024-04-02T15:57:16.267 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-04-02T15:57:16.288 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T15:57:16.297 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T15:57:16.303 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T15:57:16.303 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-02T15:57:16.305 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-04-02T15:57:16.313 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T15:57:16.319 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T15:57:16.319 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-02T15:57:16.352 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-04-02T15:57:16.364 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T15:57:16.413 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T15:57:16.427 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-02T15:57:16.427 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-02T15:57:16.428 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T15:57:16.428 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T15:57:16.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T15:57:16.433 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-02T15:57:16.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T15:57:16.434 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-02T15:57:16.478 INFO:teuthology.orchestra.run.smithi204.stdout:5.15.0-56-generic 2024-04-02T15:57:16.478 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.15.0-56-generic 2024-04-02T15:57:16.479 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2024-04-02T15:57:16.483 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-56-generic 2024-04-02T15:57:16.484 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.15.0-56-generic 2024-04-02T15:57:16.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-04-02T15:57:16.592 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-02T15:57:16.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-04-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-02T15:57:16.843 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-02T15:57:16.852 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-02T15:57:16.872 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T15:57:16.944 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-02T15:57:16.988 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-04-02T15:57:17.055 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-04-02T15:57:17.069 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-04-02T15:57:17.111 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-04-02T15:57:17.115 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-02T15:57:17.116 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-04-02T15:57:17.124 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-04-02T15:57:17.137 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-04-02T15:57:17.138 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-04-02T15:57:17.141 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-02T15:57:17.141 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-04-02T15:57:17.147 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-02T15:57:17.156 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-02T15:57:17.158 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-02T15:57:17.160 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-02T15:57:17.187 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-04-02T15:57:17.187 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-04-02T15:57:17.187 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-04-02T15:57:17.188 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-02T15:57:17.192 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-02T15:57:17.193 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-04-02T15:57:17.194 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-04-02T15:57:17.221 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-04-02T15:57:17.225 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-02T15:57:17.225 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-04-02T15:57:17.233 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-04-02T15:57:17.244 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-04-02T15:57:17.247 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-02T15:57:17.247 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-02T15:57:17.248 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-04-02T15:57:17.248 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-04-02T15:57:17.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-02T15:57:17.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-02T15:57:17.255 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-02T15:57:17.268 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-02T15:57:17.268 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-02T15:57:17.269 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-02T15:57:17.269 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-02T15:57:17.269 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-02T15:57:17.274 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-02T15:57:17.288 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-04-02T15:57:17.309 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-02T15:57:17.451 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-04-02T15:57:17.577 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-04-02T15:57:17.743 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-04-02T15:57:17.751 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-04-02T15:57:17.754 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-02T15:57:17.755 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-04-02T15:57:17.790 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-04-02T15:57:17.805 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-02T15:57:17.806 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-04-02T15:57:17.807 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-04-02T15:57:17.835 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-04-02T15:57:17.836 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-04-02T15:57:17.845 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-02T15:57:17.845 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-02T15:57:17.902 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-04-02T15:57:17.902 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-04-02T15:57:17.905 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-02T15:57:18.071 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-04-02T15:57:18.074 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-02T15:57:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-02T15:57:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-04-02T15:57:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-02T15:57:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-04-02T15:57:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-02T15:57:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-04-02T15:57:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-04-02T15:57:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-02T15:57:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-04-02T15:57:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-04-02T15:57:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-04-02T15:57:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-02T15:57:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-02T15:57:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-02T15:57:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-02T15:57:18.954 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-02T15:57:19.036 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-02T15:57:19.148 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-02T15:57:19.202 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-02T15:57:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-02T15:57:22.639 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 12.6 MB in 2s (7,042 kB/s) 2024-04-02T15:57:23.870 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 12.6 MB in 3s (4,896 kB/s) 2024-04-02T15:57:23.964 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T15:57:23.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-02T15:57:24.034 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T15:57:24.210 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-04-02T15:57:24.210 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-04-02T15:57:24.362 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T15:57:24.362 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T15:57:24.362 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T15:57:24.363 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T15:57:24.363 INFO:teuthology.orchestra.run.smithi204.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-02T15:57:24.363 INFO:teuthology.orchestra.run.smithi204.stdout: python2.7-minimal 2024-04-02T15:57:24.363 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T15:57:24.363 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-04-02T15:57:24.364 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-02T15:57:24.364 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-04-02T15:57:24.364 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-04-02T15:57:24.365 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-04-02T15:57:24.365 INFO:teuthology.orchestra.run.smithi204.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-02T15:57:24.365 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2024-04-02T15:57:24.365 INFO:teuthology.orchestra.run.smithi204.stdout: thermald 2024-04-02T15:57:24.415 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-04-02T15:57:24.415 INFO:teuthology.orchestra.run.smithi204.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-02T15:57:24.415 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-04-02T15:57:24.415 INFO:teuthology.orchestra.run.smithi204.stdout: linux-modules-extra-5.15.0-101-generic 2024-04-02T15:57:24.416 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2024-04-02T15:57:24.417 INFO:teuthology.orchestra.run.smithi204.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-02T15:57:24.595 INFO:teuthology.orchestra.run.smithi204.stdout:3 upgraded, 5 newly installed, 0 to remove and 312 not upgraded. 2024-04-02T15:57:24.596 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 113 MB of archives. 2024-04-02T15:57:24.596 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 582 MB of additional disk space will be used. 2024-04-02T15:57:24.596 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-04-02T15:57:25.159 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T15:57:25.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-02T15:57:25.242 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T15:57:25.448 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-02T15:57:25.449 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-02T15:57:25.631 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T15:57:25.632 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T15:57:25.632 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T15:57:25.632 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T15:57:25.633 INFO:teuthology.orchestra.run.smithi040.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-02T15:57:25.633 INFO:teuthology.orchestra.run.smithi040.stdout: python2.7-minimal 2024-04-02T15:57:25.633 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T15:57:25.634 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-02T15:57:25.634 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-02T15:57:25.634 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-04-02T15:57:25.634 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-04-02T15:57:25.636 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-02T15:57:25.636 INFO:teuthology.orchestra.run.smithi040.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-02T15:57:25.636 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-04-02T15:57:25.636 INFO:teuthology.orchestra.run.smithi040.stdout: thermald 2024-04-02T15:57:25.696 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-02T15:57:25.696 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-04-02T15:57:25.696 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-04-02T15:57:25.696 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-extra-5.15.0-101-generic 2024-04-02T15:57:25.697 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-04-02T15:57:25.698 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-02T15:57:25.833 INFO:teuthology.orchestra.run.smithi040.stdout:3 upgraded, 5 newly installed, 0 to remove and 312 not upgraded. 2024-04-02T15:57:25.833 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 113 MB of archives. 2024-04-02T15:57:25.833 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 582 MB of additional disk space will be used. 2024-04-02T15:57:25.833 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-04-02T15:57:26.098 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-04-02T15:57:26.428 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-04-02T15:57:26.512 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-04-02T15:57:26.660 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-04-02T15:57:27.894 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-04-02T15:57:27.895 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-04-02T15:57:27.895 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-04-02T15:57:28.385 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-04-02T15:57:28.521 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-04-02T15:57:28.562 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-04-02T15:57:28.563 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-04-02T15:57:28.563 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-04-02T15:57:28.934 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 113 MB in 3s (40.2 MB/s) 2024-04-02T15:57:29.087 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-04-02T15:57:29.334 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-04-02T15:57:29.465 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-04-02T15:57:29.870 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 113 MB in 5s (22.4 MB/s) 2024-04-02T15:57:30.042 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-04-02T15:57:31.099 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-02T15:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:31.176 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:32.188 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-02T15:57:32.193 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:32.306 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:33.312 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-04-02T15:57:33.333 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:33.426 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:33.982 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-04-02T15:57:34.003 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:34.040 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:34.424 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-04-02T15:57:34.445 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:34.531 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:34.969 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-04-02T15:57:34.990 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:35.019 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:40.467 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:40.577 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:40.860 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:40.980 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:41.300 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-04-02T15:57:41.322 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-04-02T15:57:41.366 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-02T15:57:41.421 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:41.530 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:41.830 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:41.951 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:42.254 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-04-02T15:57:42.265 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-04-02T15:57:42.304 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-02T15:57:47.271 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-04-02T15:57:47.291 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:47.321 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:48.018 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-04-02T15:57:48.055 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-04-02T15:57:48.093 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:49.917 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:50.041 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-02T15:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:50.622 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-04-02T15:57:50.759 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:51.231 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-04-02T15:57:51.356 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-04-02T15:57:51.698 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-04-02T15:57:51.802 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:51.937 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-04-02T15:57:52.066 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:52.092 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:57:52.092 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-02T15:57:52.092 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:57:52.092 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-04-02T15:57:52.245 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:53.362 INFO:teuthology.orchestra.run.smithi204.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:57:53.363 INFO:teuthology.orchestra.run.smithi204.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-02T15:57:53.363 INFO:teuthology.orchestra.run.smithi204.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:57:53.363 INFO:teuthology.orchestra.run.smithi204.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-04-02T15:57:53.502 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:54.221 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:54.886 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:55.631 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-04-02T15:57:55.765 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-04-02T15:57:55.891 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:55.954 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-02T15:57:55.955 INFO:teuthology.orchestra.run.smithi040.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:57:56.296 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-04-02T15:57:56.439 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-04-02T15:57:56.565 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-04-02T15:57:56.637 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-02T15:57:56.637 INFO:teuthology.orchestra.run.smithi204.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:58:10.494 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-02T15:58:10.494 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub' 2024-04-02T15:58:10.509 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-02T15:58:10.536 INFO:teuthology.orchestra.run.smithi040.stdout:Generating grub configuration file ... 2024-04-02T15:58:10.603 INFO:teuthology.orchestra.run.smithi204.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-02T15:58:10.603 INFO:teuthology.orchestra.run.smithi204.stdout:Sourcing file `/etc/default/grub' 2024-04-02T15:58:10.615 INFO:teuthology.orchestra.run.smithi204.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-02T15:58:10.640 INFO:teuthology.orchestra.run.smithi204.stdout:Generating grub configuration file ... 2024-04-02T15:58:10.856 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:58:10.879 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:58:10.906 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:58:10.921 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:58:11.216 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:58:11.221 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-02T15:58:11.254 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:58:11.259 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-02T15:58:11.287 INFO:teuthology.orchestra.run.smithi204.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-02T15:58:11.292 INFO:teuthology.orchestra.run.smithi204.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-02T15:58:11.327 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-02T15:58:11.333 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-02T15:58:11.456 INFO:teuthology.orchestra.run.smithi204.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-02T15:58:11.456 INFO:teuthology.orchestra.run.smithi204.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-02T15:58:11.456 INFO:teuthology.orchestra.run.smithi204.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-02T15:58:11.492 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-04-02T15:58:11.500 INFO:teuthology.orchestra.run.smithi040.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-02T15:58:11.500 INFO:teuthology.orchestra.run.smithi040.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-02T15:58:11.500 INFO:teuthology.orchestra.run.smithi040.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-02T15:58:11.545 INFO:teuthology.orchestra.run.smithi040.stdout:done 2024-04-02T15:58:11.905 DEBUG:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2024-04-02T15:58:11.972 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 21 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 5.15.0.101.98 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2024-04-02T15:58:11.973 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2024-04-02T15:58:11.974 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-04-02T15:58:11.974 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T15:58:11.974 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T15:58:11.974 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-02T15:58:11.974 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T15:58:11.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get clean 2024-04-02T15:58:12.028 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-04-02T15:58:12.050 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-04-02T15:58:12.050 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 21 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.101.98 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-04-02T15:58:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-04-02T15:58:12.052 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-04-02T15:58:12.052 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T15:58:12.052 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T15:58:12.052 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-02T15:58:12.052 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T15:58:12.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-04-02T15:58:12.126 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-04-02T15:58:12.239 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T15:58:12.251 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-02T15:58:12.255 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-02T15:58:12.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-02T15:58:12.271 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-02T15:58:12.388 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T15:58:12.389 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-02T15:58:12.398 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-02T15:58:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-02T15:58:13.458 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T15:58:13.476 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-02T15:58:13.536 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T15:58:13.751 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-04-02T15:58:13.752 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-04-02T15:58:13.763 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T15:58:13.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-02T15:58:13.835 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T15:58:13.921 INFO:teuthology.orchestra.run.smithi204.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-04-02T15:58:13.921 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T15:58:13.922 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T15:58:13.960 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 0 newly installed, 0 to remove and 312 not upgraded. 2024-04-02T15:58:13.962 DEBUG:teuthology.orchestra.run.smithi204:> dpkg -s linux-image-generic 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Package: linux-image-generic 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Status: install ok installed 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Priority: optional 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Section: kernel 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Installed-Size: 21 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Maintainer: Ubuntu Kernel Team 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: amd64 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Source: linux-meta 2024-04-02T15:58:13.974 INFO:teuthology.orchestra.run.smithi204.stdout:Version: 5.15.0.101.98 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout:Recommends: thermald 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout:Description: Generic Linux kernel image 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout: This package will always depend on the latest generic kernel image 2024-04-02T15:58:13.975 INFO:teuthology.orchestra.run.smithi204.stdout: available. 2024-04-02T15:58:13.975 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-02T15:58:14.013 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-02T15:58:14.014 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-02T15:58:14.022 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.HevQDhSHrq 2024-04-02T15:58:14.022 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HevQDhSHrq 2024-04-02T15:58:14.076 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.HevQDhSHrq 2024-04-02T15:58:14.176 INFO:teuthology.orchestra.run.smithi040.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-04-02T15:58:14.176 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T15:58:14.176 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T15:58:14.176 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T15:58:14.176 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T15:58:14.177 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T15:58:14.177 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T15:58:14.177 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T15:58:14.193 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.HevQDhSHrq is 10KB 2024-04-02T15:58:14.205 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.HevQDhSHrq 2024-04-02T15:58:14.210 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.213 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 0 newly installed, 0 to remove and 312 not upgraded. 2024-04-02T15:58:14.215 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-04-02T15:58:14.265 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T15:58:14.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.265 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 21 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.101.98 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-02T15:58:14.270 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-04-02T15:58:14.271 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-04-02T15:58:14.271 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-04-02T15:58:14.271 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-04-02T15:58:14.271 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-02T15:58:14.318 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.YD3Wd6DzTf 2024-04-02T15:58:14.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YD3Wd6DzTf 2024-04-02T15:58:14.329 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-04-02T15:58:14.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo update-grub 2024-04-02T15:58:14.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.YD3Wd6DzTf 2024-04-02T15:58:14.489 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.YD3Wd6DzTf is 10KB 2024-04-02T15:58:14.502 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.YD3Wd6DzTf 2024-04-02T15:58:14.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.566 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T15:58:14.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-02T15:58:14.636 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-04-02T15:58:14.636 DEBUG:teuthology.orchestra.run.smithi040:> sudo update-grub 2024-04-02T15:58:15.453 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub' 2024-04-02T15:58:15.454 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-02T15:58:15.468 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-04-02T15:58:15.664 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:58:15.670 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:58:15.773 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2024-04-02T15:58:15.775 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-02T15:58:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-04-02T15:58:15.906 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:58:15.911 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-02T15:58:15.952 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-02T15:58:15.957 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-02T15:58:16.038 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-04-02T15:58:16.044 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-04-02T15:58:16.116 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-02T15:58:16.116 INFO:teuthology.orchestra.run.smithi204.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-02T15:58:16.116 INFO:teuthology.orchestra.run.smithi204.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-02T15:58:16.135 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-04-02T15:58:16.137 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-04-02T15:58:16.338 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-02T15:58:16.343 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-02T15:58:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-02T15:58:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-02T15:58:16.533 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-02T15:58:16.533 INFO:teuthology.orchestra.run.smithi040.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-02T15:58:16.533 INFO:teuthology.orchestra.run.smithi040.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-02T15:58:16.546 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-04-02T15:58:16.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-04-02T15:58:46.152 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-02T15:58:46.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T15:58:46.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:58:46.563 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-02T15:58:46.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T15:58:46.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:59:20.607 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-02T15:59:29.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T15:59:29.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:59:46.587 DEBUG:teuthology.orchestra.remote:timed out 2024-04-02T15:59:48.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-02T15:59:55.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T15:59:55.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T15:59:58.623 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-04-02T16:00:00.004 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T16:00:00.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:00:00.007 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-04-02T16:00:10.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T16:00:10.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:00:10.826 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-02T16:00:11.372 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T16:00:11.372 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-04-02T16:00:11.373 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-02T16:00:11.429 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-101-generic 2024-04-02T16:00:11.430 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-04-02T16:00:11.430 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T16:00:11.430 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-02T16:00:12.431 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-02T16:00:12.431 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T16:00:12.500 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-04-02T16:00:12.547 DEBUG:teuthology.parallel:result is None 2024-04-02T16:00:15.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T16:00:15.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:00:15.266 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-02T16:00:15.788 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T16:00:15.788 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-04-02T16:00:15.788 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-04-02T16:00:15.837 INFO:teuthology.orchestra.run.smithi204.stdout:5.15.0-101-generic 2024-04-02T16:00:15.837 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-04-02T16:00:15.837 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T16:00:15.837 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-02T16:00:16.838 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-02T16:00:16.839 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-02T16:00:17.201 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-04-02T16:00:17.226 DEBUG:teuthology.parallel:result is None 2024-04-02T16:00:17.227 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T16:00:17.233 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T16:00:17.234 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:00:17.236 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T16:00:17.241 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T16:00:17.249 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T16:00:17.257 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T16:00:17.258 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:00:17.282 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T16:00:17.305 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T16:00:17.313 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T16:00:17.313 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:00:17.338 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T16:00:17.358 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:00:17.362 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:00:17.366 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:00:17.370 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T16:00:17.372 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T16:00:17.380 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T16:00:17.380 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:00:17.410 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T16:00:17.429 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T16:00:17.453 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T16:00:17.453 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:00:17.466 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T16:00:17.477 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:00:17.517 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:00:17.566 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:00:17.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:00:17.633 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T16:00:17.639 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T16:00:17.685 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:00:17.685 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T16:00:17.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-02T16:00:17.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-04-02T16:00:17.832 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T16:00:17.841 INFO:teuthology.task.internal:Starting timer... 2024-04-02T16:00:17.841 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T16:00:17.852 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T16:00:17.862 DEBUG:teuthology.task.selinux:Excluding smithi040: OS 'ubuntu' does not support SELinux 2024-04-02T16:00:17.862 DEBUG:teuthology.task.selinux:Excluding smithi204: OS 'ubuntu' does not support SELinux 2024-04-02T16:00:17.863 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T16:00:17.863 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-02T16:00:17.863 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T16:00:17.863 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T16:00:17.874 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T16:00:17.894 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T16:00:17.895 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T16:07:24.062 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-04-02T16:07:24.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T16:07:24.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:07:24.150 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-02T16:07:24.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-02T16:07:24.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T16:07:24.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T16:07:24.309 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-04-02T16:07:24.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-04-02T16:07:24.387 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T16:07:24.399 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T16:07:24.399 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:07:24.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:07:24.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T16:07:24.402 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:07:24.430 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Command line: ntpd -gq 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: ---------------------------------------------------- 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: ntp-4 is maintained by Network Time Foundation, 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: corporation. Support and training for ntp-4 are 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: available at https://www.nwtime.org/support 2024-04-02T16:07:24.431 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: ---------------------------------------------------- 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: proto: precision = 0.074 usec (-24) 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: basedate set to 2022-02-04 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: gps base set to 2022-02-06 (week 2196) 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-02T16:07:24.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: restrict ::: KOD does nothing without LIMITED. 2024-04-02T16:07:24.433 INFO:teuthology.orchestra.run.smithi040.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen and drop on 0 v6wildcard [::]:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen normally on 2 lo 127.0.0.1:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen normally on 3 ens1f0 172.21.15.40:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen normally on 4 lo [::1]:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12cc%4]:123 2024-04-02T16:07:24.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:24 ntpd[17067]: Listening on routing socket on fd #22 for interface updates 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Command line: ntpd -gq 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: ---------------------------------------------------- 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: ntp-4 is maintained by Network Time Foundation, 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: corporation. Support and training for ntp-4 are 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: available at https://www.nwtime.org/support 2024-04-02T16:07:24.455 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: ---------------------------------------------------- 2024-04-02T16:07:24.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: proto: precision = 0.063 usec (-24) 2024-04-02T16:07:24.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: basedate set to 2022-02-04 2024-04-02T16:07:24.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: gps base set to 2022-02-06 (week 2196) 2024-04-02T16:07:24.456 INFO:teuthology.orchestra.run.smithi204.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-02T16:07:24.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-02T16:07:24.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: restrict ::: KOD does nothing without LIMITED. 2024-04-02T16:07:24.457 INFO:teuthology.orchestra.run.smithi204.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen and drop on 0 v6wildcard [::]:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen normally on 2 lo 127.0.0.1:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen normally on 3 enp3s0f1 172.21.15.204:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen normally on 4 lo [::1]:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77ab%5]:123 2024-04-02T16:07:24.458 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:24 ntpd[17197]: Listening on routing socket on fd #22 for interface updates 2024-04-02T16:07:25.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:25.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:25.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:25.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:25.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:25.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:25 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:26.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:26.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:26.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:26.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:26 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:27.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:27.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:27.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:27.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:27.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:27 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:28.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:28.434 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:28 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:28.455 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:28.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:28.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:28.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:28 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:29.431 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:29.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:29 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:29.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:29.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:29 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:30.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:30.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:30 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:30.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:30 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:31.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:31 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:31.456 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:31.457 INFO:teuthology.orchestra.run.smithi204.stdout: 2 Apr 16:07:31 ntpd[17197]: ntpd: time slew -0.000674 s 2024-04-02T16:07:31.457 INFO:teuthology.orchestra.run.smithi204.stdout:ntpd: time slew -0.000674s 2024-04-02T16:07:31.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-02T16:07:31.457 INFO:teuthology.orchestra.run.smithi204.stderr: 2 Apr 16:07:31 ntpd[17197]: can't open /var/log/ntpstats/loopstats.20240402: Permission denied 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:31.503 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: can't open /var/log/ntpstats/rawstats.20240402: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: can't open /var/log/ntpstats/peerstats.20240402: Permission denied 2024-04-02T16:07:32.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2 Apr 16:07:32 ntpd[17067]: ntpd: time slew -0.000901 s 2024-04-02T16:07:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:ntpd: time slew -0.000901s 2024-04-02T16:07:32.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-02T16:07:32.433 INFO:teuthology.orchestra.run.smithi040.stderr: 2 Apr 16:07:32 ntpd[17067]: can't open /var/log/ntpstats/loopstats.20240402: Permission denied 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:32.481 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:07:32.482 INFO:teuthology.run_tasks:Running task install... 2024-04-02T16:07:32.491 DEBUG:teuthology.task.install:project ceph 2024-04-02T16:07:32.492 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T16:07:32.492 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T16:07:32.492 INFO:teuthology.task.install:Using flavor: default 2024-04-02T16:07:32.497 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T16:07:32.497 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T16:07:32.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-key list | grep Ceph 2024-04-02T16:07:32.498 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-key list | grep Ceph 2024-04-02T16:07:32.563 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-02T16:07:32.580 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-02T16:07:32.596 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-02T16:07:32.596 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph.com (release key) 2024-04-02T16:07:32.596 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-02T16:07:32.596 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-02T16:07:32.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:07:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-02T16:07:32.609 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph.com (release key) 2024-04-02T16:07:32.610 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-02T16:07:32.610 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-02T16:07:32.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:07:32.736 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default/ 2024-04-02T16:07:32.737 INFO:teuthology.task.install.deb:Package version is 19.0.0-2609-g191d963a-1jammy 2024-04-02T16:07:32.745 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default/ 2024-04-02T16:07:32.745 INFO:teuthology.task.install.deb:Package version is 19.0.0-2609-g191d963a-1jammy 2024-04-02T16:07:32.851 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:07:32.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-02T16:07:32.866 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-04-02T16:07:32.918 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:07:32.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-02T16:07:32.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-04-02T16:07:33.034 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-02T16:07:33.105 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T16:07:33.118 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-02T16:07:33.137 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-02T16:07:33.145 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy InRelease 2024-04-02T16:07:33.150 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-02T16:07:33.181 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-04-02T16:07:33.215 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy InRelease 2024-04-02T16:07:33.218 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-02T16:07:33.223 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-02T16:07:33.232 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-02T16:07:33.253 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-04-02T16:07:33.254 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-02T16:07:33.294 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-02T16:07:33.301 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-04-02T16:07:33.313 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-02T16:07:33.330 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-04-02T16:07:33.371 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-02T16:07:33.443 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 28.5 kB in 0s (70.6 kB/s) 2024-04-02T16:07:33.462 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 28.5 kB in 0s (81.7 kB/s) 2024-04-02T16:07:34.338 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T16:07:34.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2609-g191d963a-1jammy cephadm=19.0.0-2609-g191d963a-1jammy ceph-mds=19.0.0-2609-g191d963a-1jammy ceph-mgr=19.0.0-2609-g191d963a-1jammy ceph-common=19.0.0-2609-g191d963a-1jammy ceph-fuse=19.0.0-2609-g191d963a-1jammy ceph-test=19.0.0-2609-g191d963a-1jammy ceph-volume=19.0.0-2609-g191d963a-1jammy radosgw=19.0.0-2609-g191d963a-1jammy python3-rados=19.0.0-2609-g191d963a-1jammy python3-rgw=19.0.0-2609-g191d963a-1jammy python3-cephfs=19.0.0-2609-g191d963a-1jammy python3-rbd=19.0.0-2609-g191d963a-1jammy libcephfs2=19.0.0-2609-g191d963a-1jammy libcephfs-dev=19.0.0-2609-g191d963a-1jammy librados2=19.0.0-2609-g191d963a-1jammy librbd1=19.0.0-2609-g191d963a-1jammy rbd-fuse=19.0.0-2609-g191d963a-1jammy python3-cephfs=19.0.0-2609-g191d963a-1jammy cephfs-shell=19.0.0-2609-g191d963a-1jammy cephfs-top=19.0.0-2609-g191d963a-1jammy cephfs-mirror=19.0.0-2609-g191d963a-1jammy 2024-04-02T16:07:34.416 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T16:07:34.435 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T16:07:34.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2609-g191d963a-1jammy cephadm=19.0.0-2609-g191d963a-1jammy ceph-mds=19.0.0-2609-g191d963a-1jammy ceph-mgr=19.0.0-2609-g191d963a-1jammy ceph-common=19.0.0-2609-g191d963a-1jammy ceph-fuse=19.0.0-2609-g191d963a-1jammy ceph-test=19.0.0-2609-g191d963a-1jammy ceph-volume=19.0.0-2609-g191d963a-1jammy radosgw=19.0.0-2609-g191d963a-1jammy python3-rados=19.0.0-2609-g191d963a-1jammy python3-rgw=19.0.0-2609-g191d963a-1jammy python3-cephfs=19.0.0-2609-g191d963a-1jammy python3-rbd=19.0.0-2609-g191d963a-1jammy libcephfs2=19.0.0-2609-g191d963a-1jammy libcephfs-dev=19.0.0-2609-g191d963a-1jammy librados2=19.0.0-2609-g191d963a-1jammy librbd1=19.0.0-2609-g191d963a-1jammy rbd-fuse=19.0.0-2609-g191d963a-1jammy python3-cephfs=19.0.0-2609-g191d963a-1jammy cephfs-shell=19.0.0-2609-g191d963a-1jammy cephfs-top=19.0.0-2609-g191d963a-1jammy cephfs-mirror=19.0.0-2609-g191d963a-1jammy 2024-04-02T16:07:34.514 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T16:07:34.612 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-04-02T16:07:34.612 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-04-02T16:07:34.709 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-02T16:07:34.710 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-02T16:07:34.801 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T16:07:34.801 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T16:07:34.801 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T16:07:34.801 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T16:07:34.802 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T16:07:34.802 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T16:07:34.802 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-02T16:07:34.804 INFO:teuthology.orchestra.run.smithi204.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-02T16:07:34.805 INFO:teuthology.orchestra.run.smithi204.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2024-04-02T16:07:34.806 INFO:teuthology.orchestra.run.smithi204.stdout: btrfs-tools 2024-04-02T16:07:34.910 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T16:07:34.910 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T16:07:34.910 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T16:07:34.910 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T16:07:34.911 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T16:07:34.911 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T16:07:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-02T16:07:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-02T16:07:34.915 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-02T16:07:34.916 INFO:teuthology.orchestra.run.smithi204.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-02T16:07:34.917 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi204.stdout: socat unzip xmlstarlet zip 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi040.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi040.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-02T16:07:34.918 INFO:teuthology.orchestra.run.smithi040.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-04-02T16:07:34.919 INFO:teuthology.orchestra.run.smithi040.stdout: btrfs-tools 2024-04-02T16:07:34.983 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 124 newly installed, 0 to remove and 299 not upgraded. 2024-04-02T16:07:34.983 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 179 MB of archives. 2024-04-02T16:07:34.983 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-02T16:07:34.983 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-02T16:07:35.003 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-02T16:07:35.015 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-02T16:07:35.027 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-02T16:07:35.028 INFO:teuthology.orchestra.run.smithi040.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-02T16:07:35.029 INFO:teuthology.orchestra.run.smithi040.stdout: socat unzip xmlstarlet zip 2024-04-02T16:07:35.073 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-02T16:07:35.108 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-02T16:07:35.115 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-02T16:07:35.132 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-02T16:07:35.162 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2609-g191d963a-1jammy [3,554 kB] 2024-04-02T16:07:35.218 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-02T16:07:35.218 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 124 newly installed, 0 to remove and 299 not upgraded. 2024-04-02T16:07:35.218 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 179 MB of archives. 2024-04-02T16:07:35.218 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-02T16:07:35.218 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-02T16:07:35.224 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-02T16:07:35.240 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-02T16:07:35.246 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-02T16:07:35.246 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-02T16:07:35.257 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-02T16:07:35.260 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-02T16:07:35.263 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-02T16:07:35.264 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-02T16:07:35.264 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-02T16:07:35.277 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-02T16:07:35.279 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-02T16:07:35.281 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-02T16:07:35.282 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-02T16:07:35.285 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2609-g191d963a-1jammy [3,554 kB] 2024-04-02T16:07:35.300 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-02T16:07:35.302 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-02T16:07:35.302 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-02T16:07:35.302 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-02T16:07:35.302 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-02T16:07:35.303 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-02T16:07:35.303 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-02T16:07:35.320 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-02T16:07:35.320 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-02T16:07:35.324 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-02T16:07:35.324 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-02T16:07:35.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-02T16:07:35.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-02T16:07:35.341 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-02T16:07:35.341 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-02T16:07:35.342 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-02T16:07:35.343 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-02T16:07:35.349 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-02T16:07:35.360 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-02T16:07:35.361 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-02T16:07:35.367 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-02T16:07:35.367 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-02T16:07:35.380 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-02T16:07:35.382 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-02T16:07:35.383 INFO:teuthology.orchestra.run.smithi204.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-02T16:07:35.385 INFO:teuthology.orchestra.run.smithi204.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-02T16:07:35.386 INFO:teuthology.orchestra.run.smithi204.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-02T16:07:35.388 INFO:teuthology.orchestra.run.smithi204.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-02T16:07:35.401 INFO:teuthology.orchestra.run.smithi204.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-02T16:07:35.404 INFO:teuthology.orchestra.run.smithi204.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-02T16:07:35.405 INFO:teuthology.orchestra.run.smithi204.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-02T16:07:35.406 INFO:teuthology.orchestra.run.smithi204.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-02T16:07:35.421 INFO:teuthology.orchestra.run.smithi204.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-02T16:07:35.491 INFO:teuthology.orchestra.run.smithi204.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-02T16:07:35.492 INFO:teuthology.orchestra.run.smithi204.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-02T16:07:35.493 INFO:teuthology.orchestra.run.smithi204.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-02T16:07:35.494 INFO:teuthology.orchestra.run.smithi204.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-02T16:07:35.496 INFO:teuthology.orchestra.run.smithi204.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-02T16:07:35.529 INFO:teuthology.orchestra.run.smithi204.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-02T16:07:35.532 INFO:teuthology.orchestra.run.smithi204.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-02T16:07:35.532 INFO:teuthology.orchestra.run.smithi204.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-02T16:07:35.546 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-02T16:07:35.564 INFO:teuthology.orchestra.run.smithi204.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-02T16:07:35.564 INFO:teuthology.orchestra.run.smithi204.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-02T16:07:35.580 INFO:teuthology.orchestra.run.smithi204.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-02T16:07:35.581 INFO:teuthology.orchestra.run.smithi204.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-02T16:07:35.581 INFO:teuthology.orchestra.run.smithi204.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-02T16:07:35.581 INFO:teuthology.orchestra.run.smithi204.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-02T16:07:35.582 INFO:teuthology.orchestra.run.smithi204.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-02T16:07:35.583 INFO:teuthology.orchestra.run.smithi204.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-02T16:07:35.587 INFO:teuthology.orchestra.run.smithi204.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-02T16:07:35.605 INFO:teuthology.orchestra.run.smithi204.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-02T16:07:35.607 INFO:teuthology.orchestra.run.smithi204.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-02T16:07:35.608 INFO:teuthology.orchestra.run.smithi204.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-02T16:07:35.632 INFO:teuthology.orchestra.run.smithi204.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-02T16:07:35.638 INFO:teuthology.orchestra.run.smithi204.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-02T16:07:35.638 INFO:teuthology.orchestra.run.smithi204.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-02T16:07:35.639 INFO:teuthology.orchestra.run.smithi204.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-02T16:07:35.639 INFO:teuthology.orchestra.run.smithi204.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-02T16:07:35.644 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-02T16:07:35.648 INFO:teuthology.orchestra.run.smithi204.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-02T16:07:35.649 INFO:teuthology.orchestra.run.smithi204.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-02T16:07:35.657 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-02T16:07:35.664 INFO:teuthology.orchestra.run.smithi204.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-02T16:07:35.665 INFO:teuthology.orchestra.run.smithi204.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-02T16:07:35.666 INFO:teuthology.orchestra.run.smithi204.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-02T16:07:35.690 INFO:teuthology.orchestra.run.smithi204.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-02T16:07:35.693 INFO:teuthology.orchestra.run.smithi204.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-02T16:07:35.694 INFO:teuthology.orchestra.run.smithi204.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-02T16:07:35.705 INFO:teuthology.orchestra.run.smithi204.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-02T16:07:35.706 INFO:teuthology.orchestra.run.smithi204.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-02T16:07:35.706 INFO:teuthology.orchestra.run.smithi204.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-02T16:07:35.710 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-02T16:07:35.710 INFO:teuthology.orchestra.run.smithi204.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-02T16:07:35.723 INFO:teuthology.orchestra.run.smithi204.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-02T16:07:35.806 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2609-g191d963a-1jammy [3,224 kB] 2024-04-02T16:07:35.885 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-02T16:07:35.895 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-02T16:07:35.920 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-02T16:07:35.929 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-02T16:07:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2609-g191d963a-1jammy [712 kB] 2024-04-02T16:07:36.015 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2609-g191d963a-1jammy [356 kB] 2024-04-02T16:07:36.040 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2609-g191d963a-1jammy [32.8 kB] 2024-04-02T16:07:36.041 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2609-g191d963a-1jammy [180 kB] 2024-04-02T16:07:36.047 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2609-g191d963a-1jammy [66.7 kB] 2024-04-02T16:07:36.049 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2609-g191d963a-1jammy [330 kB] 2024-04-02T16:07:36.072 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2609-g191d963a-1jammy [6,659 kB] 2024-04-02T16:07:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-02T16:07:36.261 INFO:teuthology.orchestra.run.smithi204.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2609-g191d963a-1jammy [3,224 kB] 2024-04-02T16:07:36.361 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2609-g191d963a-1jammy [112 kB] 2024-04-02T16:07:36.363 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2609-g191d963a-1jammy [466 kB] 2024-04-02T16:07:36.378 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2609-g191d963a-1jammy [25.3 MB] 2024-04-02T16:07:36.671 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-02T16:07:36.687 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-02T16:07:36.703 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-02T16:07:36.711 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-02T16:07:36.712 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-02T16:07:36.728 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-02T16:07:36.766 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-02T16:07:36.779 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-02T16:07:36.783 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-02T16:07:36.784 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-02T16:07:36.813 INFO:teuthology.orchestra.run.smithi204.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2609-g191d963a-1jammy [712 kB] 2024-04-02T16:07:36.896 INFO:teuthology.orchestra.run.smithi204.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2609-g191d963a-1jammy [356 kB] 2024-04-02T16:07:36.954 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-02T16:07:36.969 INFO:teuthology.orchestra.run.smithi204.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2609-g191d963a-1jammy [32.8 kB] 2024-04-02T16:07:36.971 INFO:teuthology.orchestra.run.smithi204.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2609-g191d963a-1jammy [180 kB] 2024-04-02T16:07:36.977 INFO:teuthology.orchestra.run.smithi204.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2609-g191d963a-1jammy [66.7 kB] 2024-04-02T16:07:37.037 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-02T16:07:37.060 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-02T16:07:37.112 INFO:teuthology.orchestra.run.smithi204.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2609-g191d963a-1jammy [330 kB] 2024-04-02T16:07:37.119 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-02T16:07:37.124 INFO:teuthology.orchestra.run.smithi204.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2609-g191d963a-1jammy [6,659 kB] 2024-04-02T16:07:37.161 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-02T16:07:37.183 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-02T16:07:37.210 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-02T16:07:37.394 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-02T16:07:37.410 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-02T16:07:37.500 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-02T16:07:37.501 INFO:teuthology.orchestra.run.smithi040.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-02T16:07:37.504 INFO:teuthology.orchestra.run.smithi040.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2609-g191d963a-1jammy [5,087 kB] 2024-04-02T16:07:37.643 INFO:teuthology.orchestra.run.smithi040.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2609-g191d963a-1jammy [244 kB] 2024-04-02T16:07:37.649 INFO:teuthology.orchestra.run.smithi040.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2609-g191d963a-1jammy [124 kB] 2024-04-02T16:07:37.653 INFO:teuthology.orchestra.run.smithi040.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2609-g191d963a-1jammy [1,527 kB] 2024-04-02T16:07:37.670 INFO:teuthology.orchestra.run.smithi040.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-02T16:07:37.716 INFO:teuthology.orchestra.run.smithi040.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2609-g191d963a-1jammy [6,256 kB] 2024-04-02T16:07:37.753 INFO:teuthology.orchestra.run.smithi040.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-02T16:07:37.769 INFO:teuthology.orchestra.run.smithi040.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-02T16:07:37.955 INFO:teuthology.orchestra.run.smithi040.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-02T16:07:37.974 INFO:teuthology.orchestra.run.smithi040.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2609-g191d963a-1jammy [22.8 MB] 2024-04-02T16:07:38.006 INFO:teuthology.orchestra.run.smithi040.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-02T16:07:38.020 INFO:teuthology.orchestra.run.smithi040.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-02T16:07:38.061 INFO:teuthology.orchestra.run.smithi204.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2609-g191d963a-1jammy [112 kB] 2024-04-02T16:07:38.063 INFO:teuthology.orchestra.run.smithi204.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2609-g191d963a-1jammy [466 kB] 2024-04-02T16:07:38.126 INFO:teuthology.orchestra.run.smithi204.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2609-g191d963a-1jammy [25.3 MB] 2024-04-02T16:07:38.178 INFO:teuthology.orchestra.run.smithi040.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-02T16:07:38.179 INFO:teuthology.orchestra.run.smithi040.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-02T16:07:38.184 INFO:teuthology.orchestra.run.smithi040.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-02T16:07:38.188 INFO:teuthology.orchestra.run.smithi040.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-02T16:07:38.203 INFO:teuthology.orchestra.run.smithi040.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-02T16:07:38.388 INFO:teuthology.orchestra.run.smithi040.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-02T16:07:38.644 INFO:teuthology.orchestra.run.smithi040.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-02T16:07:38.758 INFO:teuthology.orchestra.run.smithi040.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-02T16:07:38.803 INFO:teuthology.orchestra.run.smithi040.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-02T16:07:38.884 INFO:teuthology.orchestra.run.smithi040.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-02T16:07:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-02T16:07:38.910 INFO:teuthology.orchestra.run.smithi040.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-02T16:07:39.010 INFO:teuthology.orchestra.run.smithi040.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-02T16:07:39.012 INFO:teuthology.orchestra.run.smithi040.stdout:Get:71 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2609-g191d963a-1jammy [14.0 kB] 2024-04-02T16:07:39.013 INFO:teuthology.orchestra.run.smithi040.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2609-g191d963a-1jammy [870 kB] 2024-04-02T16:07:39.030 INFO:teuthology.orchestra.run.smithi040.stdout:Get:73 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2609-g191d963a-1jammy [2,440 kB] 2024-04-02T16:07:39.073 INFO:teuthology.orchestra.run.smithi040.stdout:Get:74 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2609-g191d963a-1jammy [790 kB] 2024-04-02T16:07:39.091 INFO:teuthology.orchestra.run.smithi040.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2609-g191d963a-1jammy [147 kB] 2024-04-02T16:07:39.110 INFO:teuthology.orchestra.run.smithi040.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2609-g191d963a-1jammy [2,330 kB] 2024-04-02T16:07:39.203 INFO:teuthology.orchestra.run.smithi040.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2609-g191d963a-1jammy [8,625 kB] 2024-04-02T16:07:39.246 INFO:teuthology.orchestra.run.smithi040.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-02T16:07:39.248 INFO:teuthology.orchestra.run.smithi040.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-02T16:07:39.250 INFO:teuthology.orchestra.run.smithi040.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-02T16:07:39.415 INFO:teuthology.orchestra.run.smithi040.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-02T16:07:39.569 INFO:teuthology.orchestra.run.smithi040.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2609-g191d963a-1jammy [14.2 kB] 2024-04-02T16:07:39.569 INFO:teuthology.orchestra.run.smithi040.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2609-g191d963a-1jammy [50.3 MB] 2024-04-02T16:07:39.751 INFO:teuthology.orchestra.run.smithi040.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-02T16:07:40.148 INFO:teuthology.orchestra.run.smithi040.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-02T16:07:40.158 INFO:teuthology.orchestra.run.smithi040.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-02T16:07:40.159 INFO:teuthology.orchestra.run.smithi040.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-02T16:07:40.227 INFO:teuthology.orchestra.run.smithi040.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-02T16:07:40.228 INFO:teuthology.orchestra.run.smithi040.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-02T16:07:40.230 INFO:teuthology.orchestra.run.smithi040.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-02T16:07:40.231 INFO:teuthology.orchestra.run.smithi040.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-02T16:07:40.232 INFO:teuthology.orchestra.run.smithi040.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-02T16:07:40.398 INFO:teuthology.orchestra.run.smithi040.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-02T16:07:40.497 INFO:teuthology.orchestra.run.smithi040.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-02T16:07:40.590 INFO:teuthology.orchestra.run.smithi040.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-02T16:07:40.835 INFO:teuthology.orchestra.run.smithi040.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-02T16:07:40.878 INFO:teuthology.orchestra.run.smithi040.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-02T16:07:40.902 INFO:teuthology.orchestra.run.smithi040.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-02T16:07:40.911 INFO:teuthology.orchestra.run.smithi040.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-02T16:07:40.954 INFO:teuthology.orchestra.run.smithi040.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-02T16:07:40.979 INFO:teuthology.orchestra.run.smithi040.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-02T16:07:40.982 INFO:teuthology.orchestra.run.smithi040.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-02T16:07:40.988 INFO:teuthology.orchestra.run.smithi040.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-02T16:07:41.155 INFO:teuthology.orchestra.run.smithi040.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-02T16:07:41.684 INFO:teuthology.orchestra.run.smithi040.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-02T16:07:41.705 INFO:teuthology.orchestra.run.smithi040.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-02T16:07:41.749 INFO:teuthology.orchestra.run.smithi040.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-02T16:07:41.749 INFO:teuthology.orchestra.run.smithi040.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-02T16:07:41.752 INFO:teuthology.orchestra.run.smithi040.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-02T16:07:41.753 INFO:teuthology.orchestra.run.smithi040.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-02T16:07:41.809 INFO:teuthology.orchestra.run.smithi040.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-02T16:07:41.811 INFO:teuthology.orchestra.run.smithi040.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-02T16:07:41.889 INFO:teuthology.orchestra.run.smithi040.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-02T16:07:41.890 INFO:teuthology.orchestra.run.smithi040.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-02T16:07:41.944 INFO:teuthology.orchestra.run.smithi040.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2609-g191d963a-1jammy [121 kB] 2024-04-02T16:07:41.946 INFO:teuthology.orchestra.run.smithi040.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2609-g191d963a-1jammy [232 kB] 2024-04-02T16:07:41.951 INFO:teuthology.orchestra.run.smithi040.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2609-g191d963a-1jammy [34.0 kB] 2024-04-02T16:07:41.952 INFO:teuthology.orchestra.run.smithi040.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2609-g191d963a-1jammy [23.4 kB] 2024-04-02T16:07:41.953 INFO:teuthology.orchestra.run.smithi040.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2609-g191d963a-1jammy [38.9 kB] 2024-04-02T16:07:41.954 INFO:teuthology.orchestra.run.smithi040.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2609-g191d963a-1jammy [13.2 MB] 2024-04-02T16:07:42.050 INFO:teuthology.orchestra.run.smithi204.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2609-g191d963a-1jammy [5,087 kB] 2024-04-02T16:07:42.055 INFO:teuthology.orchestra.run.smithi040.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-02T16:07:42.399 INFO:teuthology.orchestra.run.smithi040.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2609-g191d963a-1jammy [91.6 kB] 2024-04-02T16:07:42.546 INFO:teuthology.orchestra.run.smithi040.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-02T16:07:42.561 INFO:teuthology.orchestra.run.smithi040.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-02T16:07:42.743 INFO:teuthology.orchestra.run.smithi204.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2609-g191d963a-1jammy [244 kB] 2024-04-02T16:07:42.793 INFO:teuthology.orchestra.run.smithi204.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2609-g191d963a-1jammy [124 kB] 2024-04-02T16:07:42.817 INFO:teuthology.orchestra.run.smithi204.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2609-g191d963a-1jammy [1,527 kB] 2024-04-02T16:07:43.151 INFO:teuthology.orchestra.run.smithi204.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2609-g191d963a-1jammy [6,256 kB] 2024-04-02T16:07:43.481 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 179 MB in 8s (22.9 MB/s) 2024-04-02T16:07:43.624 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-02T16:07:43.726 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-04-02T16:07:43.732 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-02T16:07:43.781 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-02T16:07:44.092 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-02T16:07:44.119 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-02T16:07:44.159 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-02T16:07:44.462 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cmd2. 2024-04-02T16:07:44.489 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-02T16:07:44.528 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-02T16:07:44.737 INFO:teuthology.orchestra.run.smithi204.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2609-g191d963a-1jammy [22.8 MB] 2024-04-02T16:07:44.881 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-02T16:07:44.908 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-02T16:07:44.990 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-02T16:07:45.344 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-02T16:07:45.370 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-02T16:07:45.410 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-02T16:07:45.688 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-02T16:07:45.715 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-02T16:07:45.746 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-02T16:07:46.192 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-02T16:07:46.218 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:07:46.266 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:07:46.678 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-02T16:07:46.705 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:07:46.745 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:07:47.056 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-02T16:07:47.083 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:07:47.114 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:07:47.476 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-02T16:07:47.503 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-02T16:07:47.543 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-02T16:07:47.862 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librados2. 2024-04-02T16:07:47.889 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../010-librados2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:47.928 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librados2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:48.373 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librbd1. 2024-04-02T16:07:48.396 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../011-librbd1_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:48.431 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librbd1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:48.868 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs2. 2024-04-02T16:07:48.895 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:48.934 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:49.113 INFO:teuthology.orchestra.run.smithi204.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2609-g191d963a-1jammy [14.0 kB] 2024-04-02T16:07:49.114 INFO:teuthology.orchestra.run.smithi204.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2609-g191d963a-1jammy [870 kB] 2024-04-02T16:07:49.199 INFO:teuthology.orchestra.run.smithi204.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2609-g191d963a-1jammy [2,440 kB] 2024-04-02T16:07:49.220 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rados. 2024-04-02T16:07:49.247 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:49.286 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rados (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:49.594 INFO:teuthology.orchestra.run.smithi204.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2609-g191d963a-1jammy [790 kB] 2024-04-02T16:07:49.623 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-02T16:07:49.649 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:07:49.689 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-argparse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:49.734 INFO:teuthology.orchestra.run.smithi204.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2609-g191d963a-1jammy [147 kB] 2024-04-02T16:07:49.775 INFO:teuthology.orchestra.run.smithi204.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2609-g191d963a-1jammy [2,330 kB] 2024-04-02T16:07:49.917 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cephfs. 2024-04-02T16:07:49.933 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:49.967 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cephfs (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:50.232 INFO:teuthology.orchestra.run.smithi204.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2609-g191d963a-1jammy [8,625 kB] 2024-04-02T16:07:50.261 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-02T16:07:50.277 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:07:50.320 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:50.648 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-prettytable. 2024-04-02T16:07:50.674 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-02T16:07:50.714 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-02T16:07:50.984 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rbd. 2024-04-02T16:07:51.002 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:51.042 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rbd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:51.337 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-02T16:07:51.364 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-02T16:07:51.395 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-02T16:07:51.681 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-02T16:07:51.708 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-02T16:07:51.747 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-02T16:07:52.059 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-02T16:07:52.086 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-02T16:07:52.125 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-02T16:07:52.455 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package lua5.1. 2024-04-02T16:07:52.482 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-02T16:07:52.520 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-02T16:07:52.585 INFO:teuthology.orchestra.run.smithi204.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2609-g191d963a-1jammy [14.2 kB] 2024-04-02T16:07:52.586 INFO:teuthology.orchestra.run.smithi204.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2609-g191d963a-1jammy [50.3 MB] 2024-04-02T16:07:53.074 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package lua-any. 2024-04-02T16:07:53.101 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-02T16:07:53.182 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-02T16:07:53.426 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package zip. 2024-04-02T16:07:53.442 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-02T16:07:53.485 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking zip (3.0-12build2) ... 2024-04-02T16:07:53.813 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package unzip. 2024-04-02T16:07:53.840 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-02T16:07:53.879 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-02T16:07:54.225 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package luarocks. 2024-04-02T16:07:54.252 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-02T16:07:54.292 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-02T16:07:54.658 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librgw2. 2024-04-02T16:07:54.685 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../027-librgw2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:54.724 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librgw2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:55.260 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rgw. 2024-04-02T16:07:55.276 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:55.318 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:55.613 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-02T16:07:55.640 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-02T16:07:55.671 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-02T16:07:55.983 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libradosstriper1. 2024-04-02T16:07:56.009 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:56.049 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libradosstriper1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:56.503 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-common. 2024-04-02T16:07:56.530 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:56.569 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:57.841 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-base. 2024-04-02T16:07:57.868 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:07:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-base (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:07:58.446 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-02T16:07:58.473 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-02T16:07:58.512 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-02T16:07:58.798 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cheroot. 2024-04-02T16:07:58.825 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-02T16:07:58.865 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-02T16:07:59.177 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-02T16:07:59.204 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-02T16:07:59.243 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-02T16:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-02T16:07:59.540 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-02T16:07:59.579 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-02T16:07:59.883 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-02T16:07:59.909 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-02T16:07:59.949 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-02T16:08:00.260 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempora. 2024-04-02T16:08:00.287 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-02T16:08:00.327 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-02T16:08:00.605 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-portend. 2024-04-02T16:08:00.634 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-02T16:08:00.663 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-02T16:08:00.949 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-02T16:08:00.976 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-02T16:08:01.016 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-02T16:08:01.352 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-02T16:08:01.379 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-02T16:08:01.419 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-02T16:08:01.706 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-natsort. 2024-04-02T16:08:01.733 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-02T16:08:01.764 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-02T16:08:02.050 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-logutils. 2024-04-02T16:08:02.077 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-02T16:08:02.116 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-02T16:08:02.420 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-02T16:08:02.447 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-02T16:08:02.486 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-02T16:08:02.764 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-mako. 2024-04-02T16:08:02.780 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-02T16:08:02.814 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-02T16:08:03.109 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-02T16:08:03.133 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-02T16:08:03.175 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-02T16:08:03.462 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-02T16:08:03.490 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-02T16:08:03.528 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-02T16:08:03.831 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webob. 2024-04-02T16:08:03.858 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-02T16:08:03.898 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-02T16:08:04.259 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-waitress. 2024-04-02T16:08:04.286 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-02T16:08:04.329 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-02T16:08:04.612 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempita. 2024-04-02T16:08:04.639 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-02T16:08:04.670 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-02T16:08:04.915 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-paste. 2024-04-02T16:08:04.942 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-02T16:08:04.981 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-02T16:08:05.000 INFO:teuthology.orchestra.run.smithi204.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2609-g191d963a-1jammy [121 kB] 2024-04-02T16:08:05.003 INFO:teuthology.orchestra.run.smithi204.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2609-g191d963a-1jammy [232 kB] 2024-04-02T16:08:05.007 INFO:teuthology.orchestra.run.smithi204.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2609-g191d963a-1jammy [34.0 kB] 2024-04-02T16:08:05.008 INFO:teuthology.orchestra.run.smithi204.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2609-g191d963a-1jammy [23.4 kB] 2024-04-02T16:08:05.009 INFO:teuthology.orchestra.run.smithi204.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2609-g191d963a-1jammy [38.9 kB] 2024-04-02T16:08:05.010 INFO:teuthology.orchestra.run.smithi204.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2609-g191d963a-1jammy [13.2 MB] 2024-04-02T16:08:05.268 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-02T16:08:05.284 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-02T16:08:05.326 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-02T16:08:05.697 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-02T16:08:05.724 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-02T16:08:05.763 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-02T16:08:06.025 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webtest. 2024-04-02T16:08:06.052 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-02T16:08:06.108 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-02T16:08:06.462 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pecan. 2024-04-02T16:08:06.490 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-02T16:08:06.528 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-02T16:08:06.694 INFO:teuthology.orchestra.run.smithi204.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2609-g191d963a-1jammy [91.6 kB] 2024-04-02T16:08:06.892 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-certifi. 2024-04-02T16:08:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-02T16:08:06.958 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-02T16:08:07.286 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-idna. 2024-04-02T16:08:07.305 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 179 MB in 32s (5,633 kB/s) 2024-04-02T16:08:07.313 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-02T16:08:07.353 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-02T16:08:07.631 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-urllib3. 2024-04-02T16:08:07.658 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-02T16:08:07.689 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-02T16:08:07.975 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests. 2024-04-02T16:08:08.003 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-02T16:08:08.033 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-02T16:08:08.326 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-02T16:08:08.428 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-04-02T16:08:08.433 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-02T16:08:08.474 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-02T16:08:08.495 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-02T16:08:08.523 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-02T16:08:08.743 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-02T16:08:08.769 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-02T16:08:08.810 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-02T16:08:08.995 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-02T16:08:09.121 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cmd2. 2024-04-02T16:08:09.147 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-02T16:08:09.188 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-02T16:08:09.524 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-02T16:08:09.550 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-02T16:08:09.633 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-02T16:08:09.648 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-02T16:08:09.665 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:09.707 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:09.978 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-02T16:08:10.004 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-02T16:08:10.044 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-02T16:08:10.129 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-02T16:08:10.156 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:10.186 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:10.364 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-02T16:08:10.390 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-02T16:08:10.431 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-02T16:08:10.506 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-02T16:08:10.533 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-02T16:08:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-02T16:08:10.818 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-02T16:08:10.844 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:08:10.893 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:08:10.976 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-openssl. 2024-04-02T16:08:11.003 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-02T16:08:11.042 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-02T16:08:11.296 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-02T16:08:11.322 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:08:11.337 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr. 2024-04-02T16:08:11.364 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:11.371 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:08:11.403 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:11.716 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-02T16:08:11.743 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-02T16:08:11.782 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:08:11.815 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mon. 2024-04-02T16:08:11.842 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:11.881 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mon (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:12.102 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-02T16:08:12.118 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-02T16:08:12.152 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-02T16:08:12.472 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librados2. 2024-04-02T16:08:12.493 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-osd. 2024-04-02T16:08:12.498 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../010-librados2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:12.521 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:12.538 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librados2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:12.560 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-osd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:12.983 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librbd1. 2024-04-02T16:08:13.009 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../011-librbd1_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:13.049 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librbd1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:13.452 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs2. 2024-04-02T16:08:13.478 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:13.522 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:13.638 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph. 2024-04-02T16:08:13.666 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../068-ceph_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:13.713 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:13.796 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rados. 2024-04-02T16:08:13.822 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:13.871 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rados (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.032 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-fuse. 2024-04-02T16:08:14.060 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:14.099 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.233 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-02T16:08:14.258 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:14.299 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-argparse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.479 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mds. 2024-04-02T16:08:14.506 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:14.526 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cephfs. 2024-04-02T16:08:14.537 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mds (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.553 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:14.593 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cephfs (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.854 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-02T16:08:14.881 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:14.920 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:14.940 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephadm. 2024-04-02T16:08:14.967 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../071-cephadm_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:15.007 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:15.241 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-prettytable. 2024-04-02T16:08:15.266 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-02T16:08:15.307 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-02T16:08:15.318 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-02T16:08:15.345 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-02T16:08:15.384 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-02T16:08:15.568 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rbd. 2024-04-02T16:08:15.583 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:15.626 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rbd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:15.712 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-02T16:08:15.739 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-02T16:08:15.787 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-02T16:08:15.979 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-02T16:08:16.005 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-02T16:08:16.045 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-02T16:08:16.290 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-02T16:08:16.305 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-02T16:08:16.348 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-02T16:08:16.643 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-02T16:08:16.669 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-02T16:08:16.710 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-02T16:08:16.817 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-babel. 2024-04-02T16:08:16.844 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-02T16:08:16.883 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-02T16:08:17.038 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua5.1. 2024-04-02T16:08:17.064 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-02T16:08:17.113 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-02T16:08:17.220 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jinja2. 2024-04-02T16:08:17.247 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-02T16:08:17.286 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-02T16:08:17.408 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-any. 2024-04-02T16:08:17.434 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-02T16:08:17.466 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-02T16:08:17.648 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-02T16:08:17.675 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:17.701 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package zip. 2024-04-02T16:08:17.714 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:17.717 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-02T16:08:17.760 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking zip (3.0-12build2) ... 2024-04-02T16:08:18.076 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-02T16:08:18.088 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package unzip. 2024-04-02T16:08:18.103 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-02T16:08:18.115 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-02T16:08:18.142 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-02T16:08:18.155 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-02T16:08:18.420 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-routes. 2024-04-02T16:08:18.436 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-02T16:08:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-02T16:08:18.659 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package luarocks. 2024-04-02T16:08:18.685 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-02T16:08:18.725 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-02T16:08:18.840 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-02T16:08:18.868 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:18.906 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:19.108 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librgw2. 2024-04-02T16:08:19.135 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../027-librgw2_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:19.175 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librgw2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:19.719 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rgw. 2024-04-02T16:08:19.734 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:19.768 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:20.111 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-02T16:08:20.114 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-02T16:08:20.128 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-02T16:08:20.140 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-02T16:08:20.169 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-02T16:08:20.180 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-02T16:08:20.442 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libradosstriper1. 2024-04-02T16:08:20.468 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:20.508 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libradosstriper1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:20.698 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-joblib. 2024-04-02T16:08:20.726 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-02T16:08:20.765 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-02T16:08:20.937 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-common. 2024-04-02T16:08:20.963 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:21.003 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:21.118 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-02T16:08:21.146 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-02T16:08:21.270 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-02T16:08:21.581 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn. 2024-04-02T16:08:21.609 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-02T16:08:21.648 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-02T16:08:22.374 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-02T16:08:22.402 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:22.425 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-base. 2024-04-02T16:08:22.440 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:22.448 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:22.500 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-base (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:23.079 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-02T16:08:23.106 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-02T16:08:23.146 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-02T16:08:23.449 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cheroot. 2024-04-02T16:08:23.468 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cachetools. 2024-04-02T16:08:23.475 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-02T16:08:23.496 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-02T16:08:23.515 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-02T16:08:23.534 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-02T16:08:23.788 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-02T16:08:23.793 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-02T16:08:23.805 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-02T16:08:23.809 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-02T16:08:23.837 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-02T16:08:23.843 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-02T16:08:24.146 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-02T16:08:24.172 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-02T16:08:24.212 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-02T16:08:24.408 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-02T16:08:24.436 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-02T16:08:24.474 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-02T16:08:24.524 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-02T16:08:24.551 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-02T16:08:24.591 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-02T16:08:24.777 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rsa. 2024-04-02T16:08:24.805 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-02T16:08:24.844 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-02T16:08:24.877 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempora. 2024-04-02T16:08:24.903 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-02T16:08:24.943 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-02T16:08:25.173 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-google-auth. 2024-04-02T16:08:25.200 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-02T16:08:25.246 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-portend. 2024-04-02T16:08:25.247 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-02T16:08:25.273 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-02T16:08:25.313 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-02T16:08:25.567 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-02T16:08:25.595 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-02T16:08:25.608 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-02T16:08:25.634 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-02T16:08:25.634 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-02T16:08:25.674 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-02T16:08:25.912 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-websocket. 2024-04-02T16:08:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-02T16:08:25.952 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-02T16:08:25.970 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-02T16:08:25.978 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-02T16:08:26.018 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-02T16:08:26.282 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-02T16:08:26.310 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-02T16:08:26.348 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-02T16:08:26.389 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-natsort. 2024-04-02T16:08:26.416 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-02T16:08:26.455 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-02T16:08:26.767 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-logutils. 2024-04-02T16:08:26.794 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-02T16:08:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-02T16:08:27.014 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-02T16:08:27.029 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:27.087 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:27.128 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-02T16:08:27.154 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-02T16:08:27.194 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-02T16:08:27.432 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-02T16:08:27.459 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-02T16:08:27.498 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-mako. 2024-04-02T16:08:27.498 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-02T16:08:27.524 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-02T16:08:27.564 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-02T16:08:27.776 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-02T16:08:27.804 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-02T16:08:27.834 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-02T16:08:27.842 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-02T16:08:27.869 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-02T16:08:27.900 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-02T16:08:28.095 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package jq. 2024-04-02T16:08:28.111 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-02T16:08:28.153 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-02T16:08:28.178 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-02T16:08:28.205 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-02T16:08:28.245 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-02T16:08:28.414 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package socat. 2024-04-02T16:08:28.431 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-02T16:08:28.473 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-02T16:08:28.565 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webob. 2024-04-02T16:08:28.591 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-02T16:08:28.631 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-02T16:08:28.814 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package xmlstarlet. 2024-04-02T16:08:28.843 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-02T16:08:28.959 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-02T16:08:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-waitress. 2024-04-02T16:08:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-02T16:08:29.188 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-02T16:08:29.331 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-test. 2024-04-02T16:08:29.358 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:29.397 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-test (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:29.504 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempita. 2024-04-02T16:08:29.530 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-02T16:08:29.570 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-02T16:08:29.832 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-paste. 2024-04-02T16:08:29.858 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-02T16:08:29.899 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-02T16:08:30.227 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-02T16:08:30.242 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-02T16:08:30.285 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-02T16:08:30.588 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-02T16:08:30.614 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-02T16:08:30.654 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-02T16:08:30.933 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webtest. 2024-04-02T16:08:30.959 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-02T16:08:30.999 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-02T16:08:31.252 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pecan. 2024-04-02T16:08:31.267 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-02T16:08:31.302 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-02T16:08:31.615 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-certifi. 2024-04-02T16:08:31.626 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-volume. 2024-04-02T16:08:31.642 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-02T16:08:31.654 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:31.682 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-02T16:08:31.684 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-volume (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:32.010 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-idna. 2024-04-02T16:08:32.030 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-02T16:08:32.036 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-02T16:08:32.058 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:32.076 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-02T16:08:32.088 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-mirror (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:32.374 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-colorama. 2024-04-02T16:08:32.387 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-urllib3. 2024-04-02T16:08:32.391 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-02T16:08:32.414 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-02T16:08:32.432 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-02T16:08:32.470 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-02T16:08:32.694 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-shell. 2024-04-02T16:08:32.722 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:32.760 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-shell (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:32.799 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests. 2024-04-02T16:08:32.825 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-02T16:08:32.865 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-02T16:08:33.029 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-top. 2024-04-02T16:08:33.046 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:33.087 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-top (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:33.135 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-02T16:08:33.150 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-02T16:08:33.185 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-02T16:08:33.307 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-02T16:08:33.323 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:33.356 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs-dev (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:33.488 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-02T16:08:33.514 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:33.554 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:33.685 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-02T16:08:33.713 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-02T16:08:33.751 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-02T16:08:34.089 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-02T16:08:34.117 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-02T16:08:34.156 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-02T16:08:34.193 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-02T16:08:34.219 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:34.259 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:34.484 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package nvme-cli. 2024-04-02T16:08:34.513 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-02T16:08:34.551 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-02T16:08:34.579 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-02T16:08:34.605 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-02T16:08:34.645 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-02T16:08:34.948 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-openssl. 2024-04-02T16:08:34.973 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package pkg-config. 2024-04-02T16:08:34.974 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-02T16:08:35.001 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-02T16:08:35.015 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-02T16:08:35.039 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-02T16:08:35.292 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-02T16:08:35.308 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-02T16:08:35.318 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr. 2024-04-02T16:08:35.345 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:35.350 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-02T16:08:35.384 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:35.745 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-02T16:08:35.773 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-02T16:08:35.811 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-02T16:08:35.812 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mon. 2024-04-02T16:08:35.839 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:35.879 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mon (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:36.106 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastescript. 2024-04-02T16:08:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-02T16:08:36.172 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-02T16:08:36.500 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-osd. 2024-04-02T16:08:36.501 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pluggy. 2024-04-02T16:08:36.526 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:36.529 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-02T16:08:36.566 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-osd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:36.584 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-02T16:08:36.895 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-psutil. 2024-04-02T16:08:36.923 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-02T16:08:36.961 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-02T16:08:37.281 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-py. 2024-04-02T16:08:37.307 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-02T16:08:37.348 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-02T16:08:37.601 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pygments. 2024-04-02T16:08:37.610 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph. 2024-04-02T16:08:37.618 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-02T16:08:37.626 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../068-ceph_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:37.651 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-02T16:08:37.669 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:37.988 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-fuse. 2024-04-02T16:08:38.015 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:38.055 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:38.072 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-02T16:08:38.100 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-02T16:08:38.138 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-02T16:08:38.417 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mds. 2024-04-02T16:08:38.443 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:38.449 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-toml. 2024-04-02T16:08:38.477 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-02T16:08:38.483 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mds (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:38.515 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-02T16:08:38.819 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pytest. 2024-04-02T16:08:38.847 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-02T16:08:38.885 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-02T16:08:38.894 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephadm. 2024-04-02T16:08:38.921 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../071-cephadm_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:38.952 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:39.222 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplejson. 2024-04-02T16:08:39.250 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-02T16:08:39.264 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-02T16:08:39.288 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-02T16:08:39.290 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-02T16:08:39.330 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-02T16:08:39.677 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-02T16:08:39.691 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-02T16:08:39.692 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-02T16:08:39.720 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-02T16:08:39.735 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-02T16:08:39.841 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-02T16:08:40.562 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package radosgw. 2024-04-02T16:08:40.590 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../122-radosgw_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:40.619 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking radosgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:40.748 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-babel. 2024-04-02T16:08:40.761 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-02T16:08:40.798 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-02T16:08:41.135 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jinja2. 2024-04-02T16:08:41.161 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-02T16:08:41.201 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-02T16:08:41.456 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package rbd-fuse. 2024-04-02T16:08:41.473 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:41.522 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking rbd-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:41.554 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-02T16:08:41.581 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:41.621 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:41.846 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-02T16:08:41.974 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-02T16:08:42.002 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-02T16:08:42.041 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-02T16:08:42.102 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-02T16:08:42.220 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-02T16:08:42.336 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-routes. 2024-04-02T16:08:42.363 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-02T16:08:42.402 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-02T16:08:42.493 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-02T16:08:42.747 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-02T16:08:42.775 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:42.813 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:42.818 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-02T16:08:43.255 INFO:teuthology.orchestra.run.smithi040.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-02T16:08:43.322 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:43.805 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user cephadm....done 2024-04-02T16:08:43.928 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-02T16:08:44.086 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-02T16:08:44.102 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-02T16:08:44.144 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-02T16:08:44.196 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-02T16:08:44.439 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-02T16:08:44.565 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-02T16:08:44.673 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-joblib. 2024-04-02T16:08:44.700 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-02T16:08:44.756 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-02T16:08:44.832 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-02T16:08:45.092 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-02T16:08:45.093 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-02T16:08:45.120 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-02T16:08:45.218 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-02T16:08:45.245 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-02T16:08:45.506 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn. 2024-04-02T16:08:45.528 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-02T16:08:45.534 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-02T16:08:45.564 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-02T16:08:45.872 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-02T16:08:46.140 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-02T16:08:46.249 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-02T16:08:46.276 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:46.315 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:46.399 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-02T16:08:46.692 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-02T16:08:46.952 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-02T16:08:47.186 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-02T16:08:47.277 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cachetools. 2024-04-02T16:08:47.304 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-02T16:08:47.343 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-02T16:08:47.429 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-02T16:08:47.596 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-02T16:08:47.624 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-02T16:08:47.663 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-02T16:08:47.680 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-argparse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:48.017 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-02T16:08:48.043 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-02T16:08:48.065 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-02T16:08:48.083 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-02T16:08:48.191 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-02T16:08:48.317 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-02T16:08:48.378 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rsa. 2024-04-02T16:08:48.405 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-02T16:08:48.444 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-02T16:08:48.584 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-02T16:08:48.694 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-02T16:08:48.757 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-google-auth. 2024-04-02T16:08:48.784 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-02T16:08:48.823 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-02T16:08:48.936 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-02T16:08:49.127 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-02T16:08:49.154 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-02T16:08:49.184 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-02T16:08:49.291 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-02T16:08:49.334 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-02T16:08:49.398 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-02T16:08:49.513 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-websocket. 2024-04-02T16:08:49.524 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-02T16:08:49.540 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-02T16:08:49.579 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-02T16:08:49.908 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-02T16:08:49.908 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-02T16:08:49.935 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-02T16:08:49.974 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-02T16:08:50.034 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-02T16:08:50.311 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-02T16:08:50.562 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-02T16:08:50.630 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-02T16:08:50.646 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:50.688 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:50.854 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up zip (3.0-12build2) ... 2024-04-02T16:08:50.989 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-02T16:08:51.033 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-02T16:08:51.060 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-02T16:08:51.099 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-02T16:08:51.387 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-02T16:08:51.414 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-02T16:08:51.453 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-02T16:08:51.682 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-02T16:08:51.722 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package jq. 2024-04-02T16:08:51.750 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-02T16:08:51.789 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-02T16:08:51.950 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-02T16:08:52.042 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package socat. 2024-04-02T16:08:52.069 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-02T16:08:52.076 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-02T16:08:52.108 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-02T16:08:52.185 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-02T16:08:52.412 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package xmlstarlet. 2024-04-02T16:08:52.439 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-02T16:08:52.478 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-02T16:08:52.478 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-02T16:08:52.841 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-test. 2024-04-02T16:08:52.854 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-02T16:08:52.861 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:52.899 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-test (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:53.106 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-02T16:08:53.532 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-02T16:08:53.833 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-02T16:08:54.143 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-02T16:08:54.477 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-02T16:08:54.712 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-02T16:08:54.838 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:55.163 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-idna (3.3-1) ... 2024-04-02T16:08:55.312 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-volume. 2024-04-02T16:08:55.339 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:55.378 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-volume (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:55.507 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-02T16:08:55.733 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-02T16:08:55.760 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:55.799 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-mirror (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:56.102 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-colorama. 2024-04-02T16:08:56.130 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-02T16:08:56.169 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-02T16:08:56.413 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-shell. 2024-04-02T16:08:56.429 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:56.463 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-shell (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:56.716 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-top. 2024-04-02T16:08:56.729 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-02T16:08:56.731 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2609-g191d963a-1jammy_all.deb ... 2024-04-02T16:08:56.774 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-top (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:57.006 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:08:57.052 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-02T16:08:57.079 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:08:57.127 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs-dev (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:08:57.140 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-02T16:08:57.408 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-02T16:08:57.446 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-02T16:08:57.474 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-02T16:08:57.512 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-02T16:08:57.534 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-02T16:08:57.643 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-02T16:08:57.842 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-02T16:08:57.870 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-02T16:08:57.908 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-02T16:08:57.953 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-02T16:08:58.196 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-02T16:08:58.238 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package nvme-cli. 2024-04-02T16:08:58.265 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-02T16:08:58.296 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-02T16:08:58.455 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:08:58.581 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-02T16:08:58.677 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package pkg-config. 2024-04-02T16:08:58.704 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-02T16:08:58.743 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-02T16:08:58.891 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-02T16:08:58.996 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-02T16:08:59.012 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-02T16:08:59.054 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-02T16:08:59.150 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-02T16:08:59.418 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-02T16:08:59.491 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-02T16:08:59.519 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-02T16:08:59.557 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-02T16:08:59.669 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-02T16:08:59.860 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastescript. 2024-04-02T16:08:59.888 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-02T16:08:59.927 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-02T16:08:59.962 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-02T16:09:00.230 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-02T16:09:00.272 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pluggy. 2024-04-02T16:09:00.300 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-02T16:09:00.338 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-02T16:09:00.358 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-02T16:09:00.600 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-psutil. 2024-04-02T16:09:00.628 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-02T16:09:00.650 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-02T16:09:00.658 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-02T16:09:00.776 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-02T16:09:00.986 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-py. 2024-04-02T16:09:01.002 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-02T16:09:01.036 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-02T16:09:01.061 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-02T16:09:01.315 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pygments. 2024-04-02T16:09:01.342 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-02T16:09:01.353 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-02T16:09:01.372 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-02T16:09:01.605 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-02T16:09:01.731 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-02T16:09:01.844 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-02T16:09:01.856 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-02T16:09:01.872 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-02T16:09:01.910 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-02T16:09:01.954 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-02T16:09:02.188 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-toml. 2024-04-02T16:09:02.204 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-02T16:09:02.208 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-02T16:09:02.238 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-02T16:09:02.334 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-02T16:09:02.533 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pytest. 2024-04-02T16:09:02.561 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-02T16:09:02.599 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-02T16:09:02.728 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-02T16:09:02.971 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-02T16:09:02.978 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplejson. 2024-04-02T16:09:03.006 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-02T16:09:03.044 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-02T16:09:03.096 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-02T16:09:03.330 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-02T16:09:03.355 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:09:03.358 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-02T16:09:03.397 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-02T16:09:03.482 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-02T16:09:03.833 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-02T16:09:04.135 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package radosgw. 2024-04-02T16:09:04.142 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-02T16:09:04.163 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../122-radosgw_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:09:04.201 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking radosgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:04.419 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-02T16:09:04.613 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-02T16:09:04.881 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-02T16:09:04.988 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package rbd-fuse. 2024-04-02T16:09:05.015 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2609-g191d963a-1jammy_amd64.deb ... 2024-04-02T16:09:05.054 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking rbd-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:05.149 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-02T16:09:05.367 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-02T16:09:05.409 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-02T16:09:05.617 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-02T16:09:05.727 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-02T16:09:05.785 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-02T16:09:05.973 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-02T16:09:06.053 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-02T16:09:06.304 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-02T16:09:06.389 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-02T16:09:06.557 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librados2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:06.692 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:06.742 INFO:teuthology.orchestra.run.smithi204.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-02T16:09:06.817 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-02T16:09:06.837 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:07.154 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:07.279 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libradosstriper1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:07.431 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-02T16:09:07.654 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user cephadm....done 2024-04-02T16:09:07.732 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librbd1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:07.778 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-02T16:09:07.867 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-modules-core (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:07.993 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:08.046 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-02T16:09:08.213 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-02T16:09:08.213 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-02T16:09:08.297 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-02T16:09:08.423 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-02T16:09:08.683 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-02T16:09:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs-dev (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:08.862 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rados (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:08.934 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-02T16:09:08.979 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librgw2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:09.051 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-02T16:09:09.105 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-top (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:09.231 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-02T16:09:09.345 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-02T16:09:09.706 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-02T16:09:09.965 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-02T16:09:10.216 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-02T16:09:10.484 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-02T16:09:10.502 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rbd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:10.711 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-02T16:09:10.962 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-02T16:09:11.003 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up rbd-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.104 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cephfs (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.222 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-02T16:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-shell (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.473 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.481 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-argparse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:11.749 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-02T16:09:11.875 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-02T16:09:12.001 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-02T16:09:12.025 INFO:teuthology.orchestra.run.smithi040.stdout:Adding group ceph....done 2024-04-02T16:09:12.252 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-02T16:09:12.376 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user ceph....done 2024-04-02T16:09:12.395 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-02T16:09:12.502 INFO:teuthology.orchestra.run.smithi040.stdout:Setting system user ceph properties....done 2024-04-02T16:09:12.517 INFO:teuthology.orchestra.run.smithi040.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-02T16:09:12.629 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-02T16:09:12.663 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-02T16:09:13.028 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-02T16:09:13.033 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-02T16:09:13.061 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-02T16:09:13.124 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-02T16:09:13.250 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-02T16:09:13.483 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-mirror (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:13.601 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-02T16:09:13.698 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:09:13.698 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:09:13.736 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-02T16:09:13.987 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-02T16:09:14.167 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-test (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:14.247 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-02T16:09:14.294 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up radosgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:14.548 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up zip (3.0-12build2) ... 2024-04-02T16:09:14.674 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-02T16:09:14.767 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:09:14.767 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:09:15.212 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-base (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:15.343 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-02T16:09:15.579 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-02T16:09:15.618 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-02T16:09:15.728 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-02T16:09:15.838 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-02T16:09:16.064 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mds (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:16.130 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-02T16:09:16.313 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-02T16:09:16.313 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-02T16:09:16.532 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-02T16:09:16.774 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-02T16:09:16.775 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-02T16:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-02T16:09:17.143 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-02T16:09:17.418 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-osd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:17.427 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-02T16:09:17.689 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-02T16:09:17.690 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-02T16:09:17.720 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-02T16:09:18.064 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-02T16:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:18.271 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:18.314 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-02T16:09:18.422 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mon (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:18.432 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-02T16:09:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-02T16:09:18.742 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-idna (3.3-1) ... 2024-04-02T16:09:19.068 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-02T16:09:19.106 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:19.249 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:19.375 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-dashboard (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:19.526 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-volume (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:19.879 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-02T16:09:19.980 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-02T16:09:20.132 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-02T16:09:20.258 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-02T16:09:20.501 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:09:20.627 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-02T16:09:20.870 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-02T16:09:20.996 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-02T16:09:21.122 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-02T16:09:21.200 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-02T16:09:21.423 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-02T16:09:21.674 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-02T16:09:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-02T16:09:21.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-02T16:09:21.917 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:09:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-04-02T16:09:22.035 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-02T16:09:22.131 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-04-02T16:09:22.132 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T16:09:22.330 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T16:09:22.331 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T16:09:22.331 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T16:09:22.331 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T16:09:22.332 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-04-02T16:09:22.332 INFO:teuthology.orchestra.run.smithi040.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-02T16:09:22.332 INFO:teuthology.orchestra.run.smithi040.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-02T16:09:22.332 INFO:teuthology.orchestra.run.smithi040.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-02T16:09:22.332 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-02T16:09:22.333 INFO:teuthology.orchestra.run.smithi040.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-02T16:09:22.334 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-04-02T16:09:22.334 INFO:teuthology.orchestra.run.smithi040.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-02T16:09:22.334 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-02T16:09:22.334 INFO:teuthology.orchestra.run.smithi040.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-02T16:09:22.338 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-02T16:09:22.376 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-04-02T16:09:22.377 INFO:teuthology.orchestra.run.smithi040.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-02T16:09:22.377 INFO:teuthology.orchestra.run.smithi040.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-02T16:09:22.378 INFO:teuthology.orchestra.run.smithi040.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-02T16:09:22.378 INFO:teuthology.orchestra.run.smithi040.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-02T16:09:22.378 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-02T16:09:22.378 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-02T16:09:22.378 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-02T16:09:22.379 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-04-02T16:09:22.380 INFO:teuthology.orchestra.run.smithi040.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-02T16:09:22.486 INFO:teuthology.orchestra.run.smithi040.stdout:4 upgraded, 39 newly installed, 0 to remove and 295 not upgraded. 2024-04-02T16:09:22.487 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 55.7 MB of archives. 2024-04-02T16:09:22.487 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-02T16:09:22.487 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-04-02T16:09:22.598 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-02T16:09:22.827 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-02T16:09:22.840 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-02T16:09:22.857 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-02T16:09:22.895 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-04-02T16:09:22.896 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-04-02T16:09:22.897 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-02T16:09:22.911 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-02T16:09:22.911 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-02T16:09:22.913 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-02T16:09:22.915 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-02T16:09:22.915 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-02T16:09:22.917 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-02T16:09:22.919 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-02T16:09:22.936 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-02T16:09:22.936 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-02T16:09:22.938 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-02T16:09:23.092 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-02T16:09:23.312 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-02T16:09:23.314 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-02T16:09:23.314 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-02T16:09:23.318 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-02T16:09:23.318 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-02T16:09:23.322 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-02T16:09:23.324 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-02T16:09:23.326 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-02T16:09:23.332 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-02T16:09:23.332 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-02T16:09:23.356 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-02T16:09:23.357 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-02T16:09:23.368 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-02T16:09:23.398 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-02T16:09:23.402 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-02T16:09:23.403 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-02T16:09:23.403 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-02T16:09:23.404 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-02T16:09:23.419 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-02T16:09:23.421 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-02T16:09:23.585 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-02T16:09:23.697 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-02T16:09:23.698 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-02T16:09:23.698 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-02T16:09:23.698 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-02T16:09:23.698 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-02T16:09:23.698 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-02T16:09:23.709 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-02T16:09:23.710 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-02T16:09:23.712 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-02T16:09:24.005 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-02T16:09:24.131 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-02T16:09:24.131 INFO:teuthology.orchestra.run.smithi040.stdout:Preconfiguring packages ... 2024-04-02T16:09:24.287 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 55.7 MB in 1s (42.1 MB/s) 2024-04-02T16:09:24.423 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-02T16:09:24.607 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-04-02T16:09:24.613 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:24.708 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-02T16:09:24.756 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:24.951 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-02T16:09:25.068 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-02T16:09:25.194 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-02T16:09:25.283 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-02T16:09:25.404 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-04-02T16:09:25.513 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-02T16:09:25.639 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-02T16:09:26.369 INFO:teuthology.orchestra.run.smithi040.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-02T16:09:26.374 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-02T16:09:26.625 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-02T16:09:26.726 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-02T16:09:26.805 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package flex. 2024-04-02T16:09:26.909 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-04-02T16:09:26.915 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:26.962 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-02T16:09:26.977 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-02T16:09:27.087 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-02T16:09:27.396 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-02T16:09:27.504 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-02T16:09:27.676 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-02T16:09:27.697 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-02T16:09:28.007 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-02T16:09:28.096 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:09:28.193 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-02T16:09:28.453 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-02T16:09:28.657 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-04-02T16:09:28.662 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:28.721 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-02T16:09:28.783 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:28.981 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-02T16:09:29.234 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:29.365 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-02T16:09:29.642 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-02T16:09:29.710 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-04-02T16:09:29.970 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-02T16:09:30.066 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package bison. 2024-04-02T16:09:30.096 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librados2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:30.173 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-04-02T16:09:30.179 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-02T16:09:30.226 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-02T16:09:30.247 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:30.373 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-02T16:09:30.619 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-02T16:09:30.647 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-02T16:09:30.651 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:30.685 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-02T16:09:30.795 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libradosstriper1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:30.920 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-02T16:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package dump. 2024-04-02T16:09:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-02T16:09:31.088 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking dump (0.4b47-3) ... 2024-04-02T16:09:31.197 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librbd1 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:31.323 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-modules-core (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:31.375 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package indent. 2024-04-02T16:09:31.392 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:31.425 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-02T16:09:31.449 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:31.643 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-02T16:09:31.643 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-02T16:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-02T16:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-02T16:09:31.711 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-02T16:09:32.074 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-02T16:09:32.102 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-02T16:09:32.125 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs-dev (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:32.140 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-02T16:09:32.243 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rados (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:32.386 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librgw2 (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:32.393 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-02T16:09:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-02T16:09:32.460 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-02T16:09:32.521 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-top (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:32.647 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-02T16:09:32.797 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-02T16:09:32.825 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:32.863 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:33.116 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-02T16:09:33.144 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:33.182 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:33.435 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjson-perl. 2024-04-02T16:09:33.452 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-02T16:09:33.494 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-02T16:09:33.763 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-02T16:09:33.780 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-02T16:09:33.806 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rbd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:33.813 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-02T16:09:33.949 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up rbd-fuse (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:34.050 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:34.151 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cephfs (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:34.260 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-shell (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:34.370 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-common (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:34.921 INFO:teuthology.orchestra.run.smithi204.stdout:Adding group ceph....done 2024-04-02T16:09:35.290 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user ceph....done 2024-04-02T16:09:35.450 INFO:teuthology.orchestra.run.smithi204.stdout:Setting system user ceph properties....done 2024-04-02T16:09:35.463 INFO:teuthology.orchestra.run.smithi204.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-02T16:09:35.558 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-02T16:09:35.587 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-02T16:09:35.602 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-02T16:09:35.625 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:35.894 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-02T16:09:35.923 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-02T16:09:35.943 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-02T16:09:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:36.272 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-02T16:09:36.300 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-02T16:09:36.330 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:09:36.422 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-mirror (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:36.630 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:09:36.630 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-02T16:09:36.650 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-02T16:09:36.678 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-02T16:09:36.708 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-02T16:09:37.096 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-02T16:09:37.098 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-test (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:37.125 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-02T16:09:37.163 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-02T16:09:37.233 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up radosgw (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:37.499 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-02T16:09:37.527 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:37.565 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:37.690 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:09:37.690 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-02T16:09:37.861 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-02T16:09:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:37.927 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:09:38.143 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-base (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:38.230 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-02T16:09:38.258 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:38.296 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:09:38.463 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-02T16:09:38.548 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-02T16:09:38.565 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-02T16:09:38.607 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:09:38.920 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mds (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:39.102 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-02T16:09:39.131 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-02T16:09:39.157 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-02T16:09:39.157 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-02T16:09:39.169 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-02T16:09:39.522 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package modemmanager. 2024-04-02T16:09:39.551 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-02T16:09:39.588 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:09:39.613 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:39.840 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-02T16:09:39.840 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-02T16:09:40.229 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager. 2024-04-02T16:09:40.257 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-02T16:09:40.304 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-02T16:09:40.323 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-osd (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:40.608 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-02T16:09:40.608 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-02T16:09:40.978 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ppp. 2024-04-02T16:09:41.006 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-02T16:09:41.108 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:41.234 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:41.268 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-02T16:09:41.369 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mon (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:41.591 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package pptp-linux. 2024-04-02T16:09:41.594 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-02T16:09:41.594 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-02T16:09:41.608 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-02T16:09:41.649 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-02T16:09:41.902 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-02T16:09:41.919 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-02T16:09:41.960 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-02T16:09:42.107 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-cephadm (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:42.324 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:42.442 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-dashboard (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:42.593 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-volume (19.0.0-2609-g191d963a-1jammy) ... 2024-04-02T16:09:42.814 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-02T16:09:42.842 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-02T16:09:42.881 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-common (238) ... 2024-04-02T16:09:42.937 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-02T16:09:43.038 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-02T16:09:43.216 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-02T16:09:43.226 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-02T16:09:43.255 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:43.309 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:43.923 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-common. 2024-04-02T16:09:43.951 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-02T16:09:44.000 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-02T16:09:44.008 INFO:teuthology.orchestra.run.smithi040.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-02T16:09:44.037 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-common (238) ... 2024-04-02T16:09:44.488 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-14. 2024-04-02T16:09:44.517 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:44.557 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:44.646 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-02T16:09:44.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-02T16:09:44.704 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-04-02T16:09:44.910 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-04-02T16:09:44.911 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-02T16:09:45.095 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-02T16:09:45.096 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-02T16:09:45.096 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-02T16:09:45.096 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-02T16:09:45.096 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-04-02T16:09:45.097 INFO:teuthology.orchestra.run.smithi204.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-02T16:09:45.097 INFO:teuthology.orchestra.run.smithi204.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-02T16:09:45.097 INFO:teuthology.orchestra.run.smithi204.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-02T16:09:45.097 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-02T16:09:45.097 INFO:teuthology.orchestra.run.smithi204.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-02T16:09:45.098 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-04-02T16:09:45.098 INFO:teuthology.orchestra.run.smithi204.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-02T16:09:45.098 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-02T16:09:45.098 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-02T16:09:45.139 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-04-02T16:09:45.139 INFO:teuthology.orchestra.run.smithi204.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-02T16:09:45.139 INFO:teuthology.orchestra.run.smithi204.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-02T16:09:45.140 INFO:teuthology.orchestra.run.smithi204.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-02T16:09:45.140 INFO:teuthology.orchestra.run.smithi204.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-02T16:09:45.140 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-02T16:09:45.140 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-02T16:09:45.140 INFO:teuthology.orchestra.run.smithi204.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-02T16:09:45.141 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2024-04-02T16:09:45.142 INFO:teuthology.orchestra.run.smithi204.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-02T16:09:45.200 INFO:teuthology.orchestra.run.smithi204.stdout:4 upgraded, 39 newly installed, 0 to remove and 295 not upgraded. 2024-04-02T16:09:45.200 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 55.7 MB of archives. 2024-04-02T16:09:45.200 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-02T16:09:45.200 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-04-02T16:09:45.360 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-02T16:09:45.369 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-02T16:09:45.411 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-04-02T16:09:45.412 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-04-02T16:09:45.413 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-02T16:09:45.423 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-02T16:09:45.424 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-02T16:09:45.426 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-02T16:09:45.427 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-02T16:09:45.427 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-02T16:09:45.429 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-02T16:09:45.433 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-02T16:09:45.451 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-02T16:09:45.452 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-02T16:09:45.453 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-02T16:09:45.835 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql. 2024-04-02T16:09:45.853 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-02T16:09:45.893 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql (14+238) ... 2024-04-02T16:09:45.904 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-02T16:09:45.908 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-02T16:09:45.908 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-02T16:09:45.912 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-02T16:09:45.912 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-02T16:09:45.919 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-02T16:09:45.921 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-02T16:09:45.932 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-02T16:09:45.932 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-02T16:09:45.963 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-02T16:09:45.963 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-02T16:09:45.979 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-02T16:09:46.038 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-02T16:09:46.048 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-02T16:09:46.049 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-02T16:09:46.050 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-02T16:09:46.050 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-02T16:09:46.077 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-02T16:09:46.080 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-02T16:09:46.146 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client. 2024-04-02T16:09:46.174 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-02T16:09:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client (14+238) ... 2024-04-02T16:09:46.409 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-02T16:09:46.409 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-02T16:09:46.409 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-02T16:09:46.410 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-02T16:09:46.410 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-02T16:09:46.410 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-02T16:09:46.424 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-02T16:09:46.424 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-02T16:09:46.456 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-02T16:09:46.485 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-02T16:09:46.522 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-02T16:09:46.830 INFO:teuthology.orchestra.run.smithi204.stdout:Preconfiguring packages ... 2024-04-02T16:09:46.967 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package tcl8.6. 2024-04-02T16:09:46.996 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-02T16:09:47.025 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-02T16:09:47.115 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 55.7 MB in 1s (43.7 MB/s) 2024-04-02T16:09:47.295 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package tcl. 2024-04-02T16:09:47.324 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-02T16:09:47.375 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-02T16:09:47.406 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-04-02T16:09:47.411 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:47.616 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:47.731 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package wpasupplicant. 2024-04-02T16:09:47.760 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-02T16:09:47.798 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-02T16:09:48.194 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-02T16:09:48.222 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-04-02T16:09:48.223 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-02T16:09:48.261 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-02T16:09:48.639 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-02T16:09:48.668 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-02T16:09:48.715 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-02T16:09:49.011 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-common (238) ... 2024-04-02T16:09:49.146 INFO:teuthology.orchestra.run.smithi204.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-02T16:09:49.220 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-02T16:09:49.346 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-02T16:09:49.447 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package flex. 2024-04-02T16:09:49.472 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:49.542 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-04-02T16:09:49.547 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:49.588 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-02T16:09:49.598 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-02T16:09:49.733 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-02T16:09:49.834 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:50.113 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-02T16:09:50.286 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-02T16:09:50.370 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-02T16:09:50.448 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-02T16:09:50.730 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:09:50.834 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-02T16:09:50.834 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-02T16:09:51.216 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-04-02T16:09:51.222 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:51.334 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-04-02T16:09:51.343 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:51.452 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-02T16:09:51.553 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:09:51.662 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-02T16:09:51.788 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-02T16:09:51.828 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-04-02T16:09:51.915 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:51.984 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-04-02T16:09:52.049 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-02T16:09:52.175 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-02T16:09:52.330 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-04-02T16:09:52.702 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package bison. 2024-04-02T16:09:52.806 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-04-02T16:09:52.812 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-02T16:09:52.863 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-02T16:09:53.022 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-02T16:09:53.272 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-02T16:09:53.298 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up dump (0.4b47-3) ... 2024-04-02T16:09:53.299 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-02T16:09:53.338 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-02T16:09:53.392 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-02T16:09:53.458 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:09:53.584 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-02T16:09:53.675 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package dump. 2024-04-02T16:09:53.703 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-02T16:09:53.710 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-02T16:09:53.741 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking dump (0.4b47-3) ... 2024-04-02T16:09:53.820 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-02T16:09:53.929 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-02T16:09:54.003 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-02T16:09:54.072 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-02T16:09:54.095 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package indent. 2024-04-02T16:09:54.112 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:09:54.145 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-02T16:09:54.198 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-02T16:09:54.398 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-02T16:09:54.400 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-02T16:09:54.426 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-02T16:09:54.473 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-02T16:09:54.526 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:54.652 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-02T16:09:54.802 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-02T16:09:54.811 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client (14+238) ... 2024-04-02T16:09:54.830 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-02T16:09:54.869 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-02T16:09:54.921 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-02T16:09:55.130 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-02T16:09:55.146 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-02T16:09:55.180 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-02T16:09:55.390 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:55.390 INFO:teuthology.orchestra.run.smithi040.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-02T16:09:55.391 INFO:teuthology.orchestra.run.smithi040.stdout:which means they are currently configured by ifupdown: 2024-04-02T16:09:55.391 INFO:teuthology.orchestra.run.smithi040.stdout:- ens1f0 2024-04-02T16:09:55.391 INFO:teuthology.orchestra.run.smithi040.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-02T16:09:55.391 INFO:teuthology.orchestra.run.smithi040.stdout:remove their configuration from /etc/network/interfaces. 2024-04-02T16:09:55.391 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:55.527 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-02T16:09:55.542 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-02T16:09:55.570 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:55.609 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:55.861 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-02T16:09:55.863 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-02T16:09:55.877 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-02T16:09:55.919 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:56.156 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjson-perl. 2024-04-02T16:09:56.172 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-02T16:09:56.186 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-02T16:09:56.206 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-02T16:09:56.501 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-02T16:09:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-02T16:09:56.567 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-02T16:09:56.740 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:56.891 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-02T16:09:57.017 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-02T16:09:57.169 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-common (238) ... 2024-04-02T16:09:58.113 INFO:teuthology.orchestra.run.smithi040.stdout:Adding user postgres to group ssl-cert 2024-04-02T16:09:58.279 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-02T16:09:58.307 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-02T16:09:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:58.345 INFO:teuthology.orchestra.run.smithi040.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-02T16:09:58.345 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:58.537 INFO:teuthology.orchestra.run.smithi040.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-02T16:09:58.537 INFO:teuthology.orchestra.run.smithi040.stdout:Removing obsolete dictionary files: 2024-04-02T16:09:58.640 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-02T16:09:58.668 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-02T16:09:58.715 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:09:58.993 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-02T16:09:59.021 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-02T16:09:59.059 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:09:59.094 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-02T16:09:59.413 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-02T16:09:59.440 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-02T16:09:59.471 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:09:59.479 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-02T16:09:59.794 INFO:teuthology.orchestra.run.smithi040.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-02T16:09:59.816 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-02T16:09:59.817 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-02T16:09:59.844 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-02T16:09:59.882 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:This user must also own the server process. 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:The database cluster will be initialized with locale "en_US". 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:The default text search configuration will be set to "english". 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout:Data page checksums are disabled. 2024-04-02T16:09:59.986 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:09:59.987 INFO:teuthology.orchestra.run.smithi040.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-02T16:09:59.987 INFO:teuthology.orchestra.run.smithi040.stdout:creating subdirectories ... ok 2024-04-02T16:09:59.987 INFO:teuthology.orchestra.run.smithi040.stdout:selecting dynamic shared memory implementation ... posix 2024-04-02T16:10:00.003 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default max_connections ... 100 2024-04-02T16:10:00.025 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default shared_buffers ... 128MB 2024-04-02T16:10:00.046 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default time zone ... Etc/UTC 2024-04-02T16:10:00.047 INFO:teuthology.orchestra.run.smithi040.stdout:creating configuration files ... ok 2024-04-02T16:10:00.173 INFO:teuthology.orchestra.run.smithi040.stdout:running bootstrap script ... ok 2024-04-02T16:10:00.219 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-02T16:10:00.246 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:10:00.285 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:00.630 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-02T16:10:00.658 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:10:00.698 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:00.994 INFO:teuthology.orchestra.run.smithi040.stdout:performing post-bootstrap initialization ... ok 2024-04-02T16:10:01.033 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-02T16:10:01.049 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:10:01.083 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:01.310 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-02T16:10:01.338 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-02T16:10:01.377 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:10:01.602 INFO:teuthology.orchestra.run.smithi040.stdout:syncing data to disk ... ok 2024-04-02T16:10:01.922 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-02T16:10:01.950 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-02T16:10:01.980 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-02T16:10:01.982 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-02T16:10:02.309 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package modemmanager. 2024-04-02T16:10:02.337 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-02T16:10:02.375 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:10:03.066 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package network-manager. 2024-04-02T16:10:03.094 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-02T16:10:03.133 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-02T16:10:03.740 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ppp. 2024-04-02T16:10:03.768 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-02T16:10:04.043 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-02T16:10:04.346 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package pptp-linux. 2024-04-02T16:10:04.362 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-02T16:10:04.404 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-02T16:10:04.682 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-02T16:10:04.710 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-02T16:10:04.757 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-02T16:10:04.782 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:04.908 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-02T16:10:05.034 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql (14+238) ... 2024-04-02T16:10:05.120 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-02T16:10:05.148 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-02T16:10:05.160 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:05.187 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client-common (238) ... 2024-04-02T16:10:05.286 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:10:05.491 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-02T16:10:05.519 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:10:05.532 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-02T16:10:05.532 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-02T16:10:05.557 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:06.089 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-02T16:10:06.137 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-common. 2024-04-02T16:10:06.166 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-02T16:10:06.181 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-02T16:10:06.222 INFO:teuthology.orchestra.run.smithi204.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-02T16:10:06.252 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-common (238) ... 2024-04-02T16:10:06.475 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-02T16:10:06.610 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-02T16:10:06.803 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-14. 2024-04-02T16:10:06.832 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-02T16:10:06.873 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:07.795 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-02T16:10:07.894 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-02T16:10:08.123 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql. 2024-04-02T16:10:08.139 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-02T16:10:08.181 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql (14+238) ... 2024-04-02T16:10:08.412 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-02T16:10:08.419 DEBUG:teuthology.parallel:result is None 2024-04-02T16:10:08.424 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client. 2024-04-02T16:10:08.441 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-02T16:10:08.483 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client (14+238) ... 2024-04-02T16:10:08.744 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-02T16:10:08.760 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-02T16:10:08.802 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-02T16:10:09.071 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package tcl8.6. 2024-04-02T16:10:09.088 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-02T16:10:09.121 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-02T16:10:09.399 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package tcl. 2024-04-02T16:10:09.427 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-02T16:10:09.470 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-02T16:10:09.818 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package wpasupplicant. 2024-04-02T16:10:09.847 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-02T16:10:09.893 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-02T16:10:10.231 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-02T16:10:10.248 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-02T16:10:10.289 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-02T16:10:10.634 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-02T16:10:10.662 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-02T16:10:10.709 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-02T16:10:11.030 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client-common (238) ... 2024-04-02T16:10:11.224 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-02T16:10:11.324 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-02T16:10:11.434 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:11.585 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-02T16:10:11.711 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-02T16:10:11.854 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:12.401 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-02T16:10:12.501 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-02T16:10:12.911 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-02T16:10:12.911 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-02T16:10:13.371 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-04-02T16:10:13.497 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-02T16:10:13.623 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-02T16:10:13.749 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-02T16:10:13.883 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-02T16:10:13.984 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-02T16:10:14.127 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-02T16:10:14.253 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-02T16:10:14.900 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-02T16:10:15.001 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up dump (0.4b47-3) ... 2024-04-02T16:10:15.095 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-02T16:10:15.211 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:10:15.337 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-02T16:10:15.463 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-02T16:10:15.589 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-02T16:10:15.715 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-02T16:10:15.806 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-02T16:10:15.867 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-02T16:10:15.976 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-02T16:10:16.128 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-02T16:10:16.253 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:10:16.388 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-02T16:10:16.556 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client (14+238) ... 2024-04-02T16:10:16.682 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-02T16:10:17.153 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout:which means they are currently configured by ifupdown: 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout:- enp3s0f1 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout:remove their configuration from /etc/network/interfaces. 2024-04-02T16:10:17.154 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:17.265 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-02T16:10:17.575 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-02T16:10:17.893 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-02T16:10:18.501 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-02T16:10:18.628 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-02T16:10:18.754 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-02T16:10:18.880 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-common (238) ... 2024-04-02T16:10:19.789 INFO:teuthology.orchestra.run.smithi204.stdout:Adding user postgres to group ssl-cert 2024-04-02T16:10:20.011 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:20.011 INFO:teuthology.orchestra.run.smithi204.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-02T16:10:20.205 INFO:teuthology.orchestra.run.smithi204.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-02T16:10:20.205 INFO:teuthology.orchestra.run.smithi204.stdout:Removing obsolete dictionary files: 2024-04-02T16:10:20.760 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-02T16:10:21.157 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-02T16:10:21.475 INFO:teuthology.orchestra.run.smithi204.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-02T16:10:21.488 INFO:teuthology.orchestra.run.smithi204.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:This user must also own the server process. 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:The database cluster will be initialized with locale "en_US". 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:The default text search configuration will be set to "english". 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:21.531 INFO:teuthology.orchestra.run.smithi204.stdout:Data page checksums are disabled. 2024-04-02T16:10:21.532 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:21.532 INFO:teuthology.orchestra.run.smithi204.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-02T16:10:21.532 INFO:teuthology.orchestra.run.smithi204.stdout:creating subdirectories ... ok 2024-04-02T16:10:21.532 INFO:teuthology.orchestra.run.smithi204.stdout:selecting dynamic shared memory implementation ... posix 2024-04-02T16:10:21.545 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default max_connections ... 100 2024-04-02T16:10:21.565 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default shared_buffers ... 128MB 2024-04-02T16:10:21.573 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default time zone ... Etc/UTC 2024-04-02T16:10:21.573 INFO:teuthology.orchestra.run.smithi204.stdout:creating configuration files ... ok 2024-04-02T16:10:21.686 INFO:teuthology.orchestra.run.smithi204.stdout:running bootstrap script ... ok 2024-04-02T16:10:22.332 INFO:teuthology.orchestra.run.smithi204.stdout:performing post-bootstrap initialization ... ok 2024-04-02T16:10:23.026 INFO:teuthology.orchestra.run.smithi204.stdout:syncing data to disk ... ok 2024-04-02T16:10:23.409 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-02T16:10:26.180 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:26.306 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-02T16:10:26.457 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql (14+238) ... 2024-04-02T16:10:26.591 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-02T16:10:26.717 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-02T16:10:26.970 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-02T16:10:26.970 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-02T16:10:27.521 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-02T16:10:27.596 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-02T16:10:27.906 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-02T16:10:28.042 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-02T16:10:29.545 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-02T16:10:29.629 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-02T16:10:30.067 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-02T16:10:30.072 DEBUG:teuthology.parallel:result is None 2024-04-02T16:10:30.073 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:10:30.293 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-query -W -f '${Version}' ceph 2024-04-02T16:10:30.314 INFO:teuthology.orchestra.run.smithi040.stdout:19.0.0-2609-g191d963a-1jammy 2024-04-02T16:10:30.314 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609-g191d963a-1jammy 2024-04-02T16:10:30.315 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609-g191d963a-1jammy is installed. 2024-04-02T16:10:30.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T16:10:30.531 DEBUG:teuthology.orchestra.run.smithi204:> dpkg-query -W -f '${Version}' ceph 2024-04-02T16:10:30.550 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-2609-g191d963a-1jammy 2024-04-02T16:10:30.550 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609-g191d963a-1jammy 2024-04-02T16:10:30.551 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609-g191d963a-1jammy is installed. 2024-04-02T16:10:30.552 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T16:10:30.552 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:30.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T16:10:30.567 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:30.567 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T16:10:30.605 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T16:10:30.605 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:30.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T16:10:30.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T16:10:30.681 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:30.681 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T16:10:30.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T16:10:30.747 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T16:10:30.748 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:30.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T16:10:30.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T16:10:30.816 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:30.816 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T16:10:30.829 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T16:10:30.883 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T16:10:30.883 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:30.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T16:10:30.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T16:10:30.952 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:30.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T16:10:30.964 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T16:10:31.020 INFO:teuthology.run_tasks:Running task ceph... 2024-04-02T16:10:31.112 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-02T16:10:31.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-04-02T16:10:31.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-04-02T16:10:31.126 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-02T16:10:31.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T16:10:31.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-02T16:10:31.184 INFO:tasks.ceph:Creating extra log directories... 2024-04-02T16:10:31.184 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T16:10:31.226 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-02T16:10:31.250 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-02T16:10:31.250 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-02T16:10:31.250 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-02T16:10:31.250 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:31.250 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-02T16:10:31.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-02T16:10:31.360 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:31.361 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-02T16:10:31.366 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-02T16:10:31.447 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-02T16:10:31.447 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-02T16:10:31.447 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686', 'branch': 'wip-batrick-testing-20240402.004512', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7635686', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240402.004512'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240402.004512', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2763, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'sleep_before_teardown': 0, 'subset': '318/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240402.004512', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRkQdXxPy/y2GsjXPcZmsdMRTTfTw3lb43j9ELZzcr0kLRrXHgO+ltSrf0OT/QX3M+AIKGCfHZzUFk8gB5NL9g=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPVksMOgDS4ZluSQ96fhgGWRK7u4t7F6s4evIu+rAK70Q3Q9d12fWJFVIIu/bU7DyogNnL260EZII3oxw/t1W8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-04-02_11:52:43', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-02T16:10:31.448 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T16:10:31.450 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-02T16:10:31.489 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T16:10:31.497 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-04-02T16:10:31.544 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:31.544 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T16:10:31.560 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T16:10:31.560 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 871 Links: 1 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-02 16:04:03.874090619 +0000 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-02 16:04:03.630092798 +0000 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-02 16:04:03.630092798 +0000 2024-04-02T16:10:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-02T16:10:31.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T16:10:31.663 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-02T16:10:31.663 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-02T16:10:31.663 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000419686 s, 1.2 MB/s 2024-04-02T16:10:31.665 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T16:10:31.714 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-02T16:10:31.761 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T16:10:31.761 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:31.761 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-04-02T16:10:31.761 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:31.761 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-02 16:04:04.386086046 +0000 2024-04-02T16:10:31.762 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-02 16:04:04.114088475 +0000 2024-04-02T16:10:31.762 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-02 16:04:04.114088475 +0000 2024-04-02T16:10:31.762 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-02T16:10:31.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T16:10:31.815 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-02T16:10:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-02T16:10:31.816 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000402293 s, 1.3 MB/s 2024-04-02T16:10:31.817 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T16:10:31.866 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-02T16:10:31.912 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-02 16:04:04.918081294 +0000 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-02 16:04:04.622083938 +0000 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-02 16:04:04.622083938 +0000 2024-04-02T16:10:31.913 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-02T16:10:31.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T16:10:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-02T16:10:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-02T16:10:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000440413 s, 1.2 MB/s 2024-04-02T16:10:31.969 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T16:10:32.020 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-02T16:10:32.068 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T16:10:32.068 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:32.068 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-04-02T16:10:32.069 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:32.069 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-02 16:04:05.414076864 +0000 2024-04-02T16:10:32.069 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-02 16:04:05.150079221 +0000 2024-04-02T16:10:32.069 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-02 16:04:05.150079221 +0000 2024-04-02T16:10:32.069 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-02T16:10:32.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T16:10:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-02T16:10:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-02T16:10:32.123 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000369222 s, 1.4 MB/s 2024-04-02T16:10:32.124 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T16:10:32.174 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:32.175 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:32.175 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T16:10:32.180 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T16:10:32.181 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 16:04:01.919739479 +0000 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 16:04:01.675741658 +0000 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 16:04:01.675741658 +0000 2024-04-02T16:10:32.230 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-04-02T16:10:32.231 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T16:10:32.283 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T16:10:32.284 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T16:10:32.284 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000348071 s, 1.5 MB/s 2024-04-02T16:10:32.285 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T16:10:32.338 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-04-02T16:10:32.384 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T16:10:32.384 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 16:04:02.475734513 +0000 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 16:04:02.179737157 +0000 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 16:04:02.179737157 +0000 2024-04-02T16:10:32.385 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-04-02T16:10:32.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T16:10:32.439 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T16:10:32.439 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T16:10:32.439 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000340722 s, 1.5 MB/s 2024-04-02T16:10:32.441 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T16:10:32.494 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-04-02T16:10:32.540 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 16:04:03.003729796 +0000 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 16:04:02.711732404 +0000 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 16:04:02.711732404 +0000 2024-04-02T16:10:32.541 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-04-02T16:10:32.541 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T16:10:32.600 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T16:10:32.600 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T16:10:32.600 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000275881 s, 1.9 MB/s 2024-04-02T16:10:32.601 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T16:10:32.650 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-04-02 16:04:03.483725509 +0000 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-04-02 16:04:03.215727903 +0000 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-04-02 16:04:03.215727903 +0000 2024-04-02T16:10:32.697 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2024-04-02T16:10:32.698 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T16:10:32.751 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-04-02T16:10:32.751 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-04-02T16:10:32.751 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000370426 s, 1.4 MB/s 2024-04-02T16:10:32.752 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T16:10:32.802 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:32.802 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-02T16:10:32.802 INFO:tasks.ceph:Generating config... 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] debug client = 20 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] debug ms = 1 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] pid file = 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-02T16:10:32.804 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-02T16:10:32.804 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-02T16:10:32.804 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-02T16:10:32.804 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-02T16:10:32.805 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-02T16:10:32.806 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-02T16:10:32.806 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-02T16:10:32.806 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-02T16:10:32.806 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-02T16:10:32.806 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-02T16:10:32.806 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-02T16:10:32.806 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-02T16:10:32.806 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-02T16:10:32.806 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-02T16:10:32.807 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-02T16:10:32.808 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-02T16:10:32.808 INFO:tasks.ceph:Setting up mon.a... 2024-04-02T16:10:32.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-02T16:10:32.844 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-04-02T16:10:32.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-02T16:10:32.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:10:32.988 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-04-02T16:10:32.989 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-02T16:10:32.989 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:32.990 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T16:10:33.039 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid f7c926e5-333e-4244-9af2-414c6ebf5077 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:fsid f7c926e5-333e-4244-9af2-414c6ebf5077 2024-04-02T16:10:33.127 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-04-02T16:10:33.128183+0000 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-04-02T16:10:33.128183+0000 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-04-02T16:10:33.128 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-02T16:10:33.129 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-02T16:10:33.177 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f7c926e5-333e-4244-9af2-414c6ebf5077... 2024-04-02T16:10:33.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T16:10:33.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-02T16:10:33.251 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-04-02T16:10:33.251 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-04-02T16:10:33.251 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T16:10:33.251 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-04-02T16:10:33.251 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.252 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-04-02T16:10:33.253 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.40,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = f7c926e5-333e-4244-9af2-414c6ebf5077 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.254 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-04-02T16:10:33.255 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.256 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore allocator = bitmap 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 1/20 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 1/20 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 4/10 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-04-02T16:10:33.257 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-04-02T16:10:33.258 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T16:10:33.259 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-04-02T16:10:33.260 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-04-02T16:10:33.261 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-04-02T16:10:33.262 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.263 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-04-02T16:10:33.264 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-04-02T16:10:33.265 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = f7c926e5-333e-4244-9af2-414c6ebf5077 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.266 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-04-02T16:10:33.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore allocator = bitmap 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 1/20 2024-04-02T16:10:33.268 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 1/20 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 4/10 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-02T16:10:33.269 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-04-02T16:10:33.270 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-04-02T16:10:33.271 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: mon down mkfs grace = 300 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-02T16:10:33.272 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-04-02T16:10:33.273 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-04-02T16:10:33.274 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-04-02T16:10:33.274 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-04-02T16:10:33.274 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-04-02T16:10:33.275 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-02T16:10:33.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-02T16:10:33.373 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-02T16:10:33.373 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:33.373 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-02T16:10:33.378 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:33.378 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-02T16:10:33.423 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:10:33.423 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:33.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T16:10:33.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:10:33.490 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:10:33.490 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:33.536 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:10:33.537 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:33.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-02T16:10:33.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-02T16:10:33.559 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:10:33.560 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:33.608 INFO:tasks.ceph:Setting up mon nodes... 2024-04-02T16:10:33.609 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-02T16:10:33.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-02T16:10:33.655 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-02T16:10:33.658 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T16:10:33.701 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-02T16:10:33.710 INFO:tasks.ceph:Setting up mds nodes... 2024-04-02T16:10:33.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-02T16:10:33.755 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-02T16:10:33.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-02T16:10:33.811 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-02T16:10:33.854 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-02T16:10:33.857 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-02T16:10:33.912 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-02T16:10:33.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T16:10:33.949 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-02T16:10:33.961 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-02T16:10:33.961 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-02T16:10:33.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-02T16:10:34.016 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:34.016 INFO:tasks.ceph:role: osd.0 2024-04-02T16:10:34.016 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:10:34.016 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T16:10:34.080 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:34.080 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:34.080 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:34.081 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-02T16:10:34.535 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-02T16:10:34.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-02T16:10:34.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-02T16:10:34.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:34.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-02T16:10:34.698 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:34.698 INFO:tasks.ceph:role: osd.1 2024-04-02T16:10:34.698 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:10:34.699 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T16:10:34.760 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:34.761 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:35.231 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-02T16:10:35.232 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-02T16:10:35.233 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-02T16:10:35.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-02T16:10:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:35.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:35.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-02T16:10:35.457 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:35.457 INFO:tasks.ceph:role: osd.2 2024-04-02T16:10:35.457 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:10:35.457 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:35.520 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:35.521 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-02T16:10:35.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-02T16:10:35.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-02T16:10:36.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-02T16:10:36.152 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:36.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:36.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-02T16:10:36.208 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:36.208 INFO:tasks.ceph:role: osd.3 2024-04-02T16:10:36.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:10:36.208 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:36.273 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:36.699 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-04-02T16:10:36.701 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-04-02T16:10:36.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-02T16:10:36.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-02T16:10:36.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:36.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:36.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.913+0000 7f383b29b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-02T16:10:36.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.917+0000 7f383b29b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-02T16:10:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.917+0000 7f383b29b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:10:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.917+0000 7f383b29b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:10:36.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.917+0000 7f383b29b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-02T16:10:36.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:36.925+0000 7f383b29b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-02T16:10:39.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-02T16:10:39.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.017+0000 7fcc617067c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-02T16:10:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.021+0000 7fcc617067c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-02T16:10:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.021+0000 7fcc617067c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:10:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.021+0000 7fcc617067c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:10:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.021+0000 7fcc617067c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-02T16:10:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:40.021+0000 7fcc617067c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-02T16:10:43.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-02T16:10:43.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-02T16:10:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-02T16:10:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:10:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:10:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-02T16:10:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:43.161+0000 7f85d2de77c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-02T16:10:46.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-02T16:10:46.201 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:46.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-02T16:10:46.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-02T16:10:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:10:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:10:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-02T16:10:46.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:10:46.281+0000 7fa02a3d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-02T16:10:49.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-02T16:10:49.313 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-02T16:10:49.328 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:49.328 INFO:tasks.ceph:role: osd.4 2024-04-02T16:10:49.328 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:10:49.328 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:49.392 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:49.393 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:49.393 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:49.852 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-02T16:10:49.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-02T16:10:49.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-02T16:10:49.954 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-02T16:10:49.965 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:49.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:49.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-02T16:10:50.018 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:50.019 INFO:tasks.ceph:role: osd.5 2024-04-02T16:10:50.019 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:10:50.019 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:50.081 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:50.545 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-02T16:10:50.547 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-02T16:10:50.547 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-02T16:10:50.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-02T16:10:50.699 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:50.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:50.700 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-02T16:10:50.756 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:50.756 INFO:tasks.ceph:role: osd.6 2024-04-02T16:10:50.756 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:10:50.756 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:50.820 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:50.821 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:51.307 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-02T16:10:51.309 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-02T16:10:51.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-02T16:10:51.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-02T16:10:51.460 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:51.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:51.461 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-02T16:10:51.516 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-02T16:10:51.516 INFO:tasks.ceph:role: osd.7 2024-04-02T16:10:51.516 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:10:51.517 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-02T16:10:51.580 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-02T16:10:51.580 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-02T16:10:51.580 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-02T16:10:51.581 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-02T16:10:52.016 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-04-02T16:10:52.018 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-04-02T16:10:52.018 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-02T16:10:52.115 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-02T16:10:52.124 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-04-02T16:10:52.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:10:52.125 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:52.228 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.221+0000 7fc2985fd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-02T16:10:52.232 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.225+0000 7fc2985fd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-02T16:10:52.232 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.225+0000 7fc2985fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:10:52.232 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.225+0000 7fc2985fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:10:52.232 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.225+0000 7fc2985fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-02T16:10:52.235 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:52.229+0000 7fc2985fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-02T16:10:55.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-02T16:10:55.237 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:55.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-02T16:10:55.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-02T16:10:55.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:10:55.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:10:55.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-02T16:10:55.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:55.313+0000 7fd2f6c227c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-02T16:10:58.308 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-02T16:10:58.364 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:10:58.447 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.441+0000 7f4ef18967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-02T16:10:58.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.441+0000 7f4ef18967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-02T16:10:58.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.441+0000 7f4ef18967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:10:58.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.441+0000 7f4ef18967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:10:58.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.441+0000 7f4ef18967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-02T16:10:58.448 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:10:58.445+0000 7f4ef18967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-02T16:11:01.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-02T16:11:01.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:11:01.452 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:11:01.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:11:01.561 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-02T16:11:01.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-02T16:11:01.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:11:01.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:11:01.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-02T16:11:01.562 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:01.557+0000 7f479ee357c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-02T16:11:04.547 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-02T16:11:04.609 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-02T16:11:04.609 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-02T16:11:04.624 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-02T16:11:04.680 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-02T16:11:04.737 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-02T16:11:04.795 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.795 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-02T16:11:04.851 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:04.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-02T16:11:04.908 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:04.908 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-02T16:11:04.920 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:04.920 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-02T16:11:04.976 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:04.976 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-02T16:11:05.031 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:05.031 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-02T16:11:05.087 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:05.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-02T16:11:05.141 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:05.141 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-02T16:11:05.195 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:05.195 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-02T16:11:05.201 INFO:tasks.ceph:Adding keys to all mons... 2024-04-02T16:11:05.201 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-02T16:11:05.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-02T16:11:05.254 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-04-02T16:11:05.254 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxmC8MbJxAACP7SD6VdBN3rfrbHid7szw== 2024-04-02T16:11:05.254 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-04-02T16:11:05.254 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxmFZ4ZLRAAg9UAvB41KyNBsCRcvn5cZQ== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD8LQxmU7rgNhAAFS4pWl9VA3B04Dl7RnTg4A== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAALgxmjsN6ARAA0011kcPPwUGqnylR78uhxA== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQADLgxmkgLVCRAAhUqJntUebAs0li6gWAG4eg== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAGLgxm4B8GERAAOXhjH8GXhH7jNNZl9+aRRw== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxmMyrbKRAAsjswzcBmDGKDF007lGbGKw== 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-04-02T16:11:05.255 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxm5UwBMxAA3LvKPHoC1VUToQkOOiE/TA== 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAMLgxmKeGnDRAAQXoRx/Cm6ucbtKRfz8rA6A== 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAPLgxmkcgFExAAS94XrJ6sYuw2Jct16snpSQ== 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQASLgxmSde+GhAAvCSikfKuv2IlJtuC0UkOww== 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAVLgxm5+2KIRAAq3Mzsnq5IHzVM2PJd4bkjg== 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-02T16:11:05.256 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxm3/mnOBAA99WD/i8maqhym9P/RBkKdw== 2024-04-02T16:11:05.258 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-04-02T16:11:05.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD5LQxmC8MbJxAACP7SD6VdBN3rfrbHid7szw== 2024-04-02T16:11:05.258 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-04-02T16:11:05.258 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD5LQxmFZ4ZLRAAg9UAvB41KyNBsCRcvn5cZQ== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD8LQxmU7rgNhAAFS4pWl9VA3B04Dl7RnTg4A== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAALgxmjsN6ARAA0011kcPPwUGqnylR78uhxA== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQADLgxmkgLVCRAAhUqJntUebAs0li6gWAG4eg== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAGLgxm4B8GERAAOXhjH8GXhH7jNNZl9+aRRw== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD5LQxmMyrbKRAAsjswzcBmDGKDF007lGbGKw== 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-04-02T16:11:05.259 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD5LQxm5UwBMxAA3LvKPHoC1VUToQkOOiE/TA== 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAMLgxmKeGnDRAAQXoRx/Cm6ucbtKRfz8rA6A== 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAPLgxmkcgFExAAS94XrJ6sYuw2Jct16snpSQ== 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQASLgxmSde+GhAAvCSikfKuv2IlJtuC0UkOww== 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQAVLgxm5+2KIRAAq3Mzsnq5IHzVM2PJd4bkjg== 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-04-02T16:11:05.260 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQD5LQxm3/mnOBAA99WD/i8maqhym9P/RBkKdw== 2024-04-02T16:11:05.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:11:05.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:11:05.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:11:05.353 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:11:05.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.525 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.609 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:11:05.740 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-02T16:11:05.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:11:05.825 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-02T16:11:05.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.903 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:05.989 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:06.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:06.072 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:06.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:06.157 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-02T16:11:06.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:11:06.241 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-02T16:11:06.324 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-02T16:11:06.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-02T16:11:06.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:11:06.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-02T16:11:06.446 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-02T16:11:06.460 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:11:06.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-02T16:11:06.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-02T16:11:06.619 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-02T16:11:06.715 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-02T16:11:06.726 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:11:06.728 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-02T16:11:06.772 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-02T16:11:06.772 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-02T16:11:06.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-02T16:11:06.777 INFO:tasks.ceph.mon.a:Started 2024-04-02T16:11:06.777 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-02T16:11:06.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-02T16:11:06.817 INFO:tasks.ceph.mon.b:Started 2024-04-02T16:11:06.817 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-02T16:11:06.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-02T16:11:06.820 INFO:tasks.ceph.mon.c:Started 2024-04-02T16:11:06.820 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-02T16:11:06.820 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-02T16:11:06.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-02T16:11:06.822 INFO:tasks.ceph.mgr.y:Started 2024-04-02T16:11:06.822 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-02T16:11:06.822 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-02T16:11:06.824 INFO:tasks.ceph.mgr.x:Started 2024-04-02T16:11:06.824 INFO:tasks.ceph:Setting crush tunables to default 2024-04-02T16:11:06.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-04-02T16:11:06.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 Processor -- start 2024-04-02T16:11:06.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 -- start start 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8414c9f0 con 0x7f2b8414f570 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2b840b1800 con 0x7f2b840b3720 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b8880a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8414bf10 con 0x7f2b84072770 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b82d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:06.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b82d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43368/0 (socket says 172.21.15.40:43368) 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.937+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 learned_addr learned my addr 172.21.15.40:0/3791950250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 msgr2=0x7f2b841563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 msgr2=0x7f2b841563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b8259c640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 msgr2=0x7f2b8414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b8259c640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 msgr2=0x7f2b8414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:11:06.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:06.941+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T16:11:07.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 msgr2=0x7f2b841563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 msgr2=0x7f2b841563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b8259c640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 msgr2=0x7f2b8414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b8259c640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 msgr2=0x7f2b8414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T16:11:07.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.145+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T16:11:07.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294670461 0 0) 0x7f2b8414c9f0 con 0x7f2b8414f570 2024-04-02T16:11:07.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b6c003530 con 0x7f2b8414f570 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2094069997 0 0) 0x7f2b6c003530 con 0x7f2b8414f570 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 msgr2=0x7f2b8414f030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.204:3300/0 conn(0x7f2b84072770 0x7f2b8414f030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 msgr2=0x7f2b841563c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 --2- 172.21.15.40:0/3791950250 >> v2:172.21.15.40:3300/0 conn(0x7f2b840b3720 0x7f2b841563c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8414c1f0 con 0x7f2b8414f570 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f2b78001680 con 0x7f2b8414f570 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 >> v1:172.21.15.40:6789/0 conn(0x7f2b8414f570 legacy=0x7f2b840b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6c003a10 0x7f2b6c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b6c0052f0 0x7f2b6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840b1800 con 0x7f2b6c004810 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8414bf10 con 0x7f2b6c0052f0 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c00c240 con 0x7f2b6c003a10 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.185+0000 7f2b82d9d640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b6c0052f0 0x7f2b6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b8259c640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6c003a10 0x7f2b6c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b81d9b640 1 -- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6c003a10 msgr2=0x7f2b6c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6c003a10 0x7f2b6c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b81d9b640 1 -- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b6c0052f0 msgr2=0x7f2b6c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b6c0052f0 0x7f2b6c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b82d9d640 1 -- 172.21.15.40:0/3791950250 reap_dead start 2024-04-02T16:11:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.189+0000 7f2b81d9b640 1 -- 172.21.15.40:0/3791950250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c003530 con 0x7f2b6c004810 2024-04-02T16:11:07.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.193+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b6c004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b78007f50 tx=0x7f2b7801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbe52b189567292 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:07.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.193+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780b2070 con 0x7f2b6c004810 2024-04-02T16:11:07.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.193+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b780a5860 con 0x7f2b6c004810 2024-04-02T16:11:07.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.193+0000 7f2b8159a640 1 -- 172.21.15.40:0/3791950250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78007ba0 con 0x7f2b6c004810 2024-04-02T16:11:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 -- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 msgr2=0x7f2b6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b6c004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b78007f50 tx=0x7f2b7801ca30 comp rx=0 tx=0).stop 2024-04-02T16:11:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 -- 172.21.15.40:0/3791950250 shutdown_connections 2024-04-02T16:11:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 --2- 172.21.15.40:0/3791950250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b6c004c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 -- 172.21.15.40:0/3791950250 >> 172.21.15.40:0/3791950250 conn(0x7f2b840b7f50 msgr2=0x7f2b840b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:07.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 -- 172.21.15.40:0/3791950250 shutdown_connections 2024-04-02T16:11:07.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.197+0000 7f2b8880a640 1 -- 172.21.15.40:0/3791950250 wait complete. 2024-04-02T16:11:07.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 Processor -- start 2024-04-02T16:11:07.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- start start 2024-04-02T16:11:07.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 0x7f2b840b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b841e48d0 0x7f2b841e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2b840b1450 con 0x7f2b6c004810 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8414bac0 con 0x7f2b841e48d0 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2b841daf80 con 0x7f2b8414e720 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43424/0 (socket says 172.21.15.40:43424) 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 -- 172.21.15.40:0/2382739495 learned_addr learned my addr 172.21.15.40:0/2382739495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 0x7f2b840b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 -- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 msgr2=0x7f2b840b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 0x7f2b840b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 -- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b841e48d0 msgr2=0x7f2b841e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b841e48d0 0x7f2b841e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 -- 172.21.15.40:0/2382739495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8414ebb0 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b81d9b640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 0x7f2b840b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8259c640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2b74004650 tx=0x7f2b74007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7401a020 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74016660 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74016a10 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b840b4c20 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.201+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b840b5140 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.205+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2b7401d020 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.205+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2b740153c0 con 0x7f2b6c004810 2024-04-02T16:11:07.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.205+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84068d00 con 0x7f2b6c004810 2024-04-02T16:11:07.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.205+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2b74024030 con 0x7f2b6c004810 2024-04-02T16:11:07.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.273+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2b840b1070 con 0x7f2b6c004810 2024-04-02T16:11:07.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.273+0000 7f2b72ffd640 1 -- 172.21.15.40:0/2382739495 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f2b7401f400 con 0x7f2b6c004810 2024-04-02T16:11:07.279 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 msgr2=0x7f2b841da480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2b74004650 tx=0x7f2b74007700 comp rx=0 tx=0).stop 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 shutdown_connections 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b8414e720 0x7f2b840b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b841e48d0 0x7f2b841e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 --2- 172.21.15.40:0/2382739495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b6c004810 0x7f2b841da480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 >> 172.21.15.40:0/2382739495 conn(0x7f2b840b7f50 msgr2=0x7f2b84149a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 shutdown_connections 2024-04-02T16:11:07.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:07.277+0000 7f2b8880a640 1 -- 172.21.15.40:0/2382739495 wait complete. 2024-04-02T16:11:07.298 INFO:tasks.ceph:check_enable_crimson: False 2024-04-02T16:11:07.298 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-02T16:11:07.298 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:07.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-02T16:11:07.310 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:07.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-02T16:11:07.367 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:07.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-02T16:11:07.389 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:07.385+0000 7f3a5c26d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T16:11:07.391 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:07.385+0000 7f75cedb8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T16:11:07.399 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:07.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-02T16:11:07.455 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:07.455 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-02T16:11:07.467 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:07.468 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-02T16:11:07.524 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:07.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-02T16:11:07.580 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-04-02T16:11:07.580 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-02T16:11:07.636 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec 0 2024-04-02T16:11:07.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.797+0000 7f4310646640 1 Processor -- start 2024-04-02T16:11:07.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.797+0000 7f4310646640 1 -- start start 2024-04-02T16:11:07.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f4310646640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f430806e1e0 0x7f4308153000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f4310646640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43080b6ff0 con 0x7f430802c900 2024-04-02T16:11:07.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f4310646640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f430802cce0 con 0x7f43080b5e70 2024-04-02T16:11:07.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f4310646640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43080b7600 con 0x7f430806e1e0 2024-04-02T16:11:07.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430dbe1640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f43080b5e70 0x7f430806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43290/0 (socket says 172.21.15.204:43290) 2024-04-02T16:11:07.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430dbe1640 1 -- 172.21.15.204:0/593005984 learned_addr learned my addr 172.21.15.204:0/593005984 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-02T16:11:07.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430ebe3640 1 --2- 172.21.15.204:0/593005984 >> v2:172.21.15.40:3300/0 conn(0x7f430806e1e0 0x7f4308153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1159926111 0 0) 0x7f430802cce0 con 0x7f43080b5e70 2024-04-02T16:11:07.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42f4003530 con 0x7f43080b5e70 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4216287 0 0) 0x7f42f4003530 con 0x7f43080b5e70 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 >> v2:172.21.15.40:3300/0 conn(0x7f430806e1e0 msgr2=0x7f4308153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 --2- 172.21.15.204:0/593005984 >> v2:172.21.15.40:3300/0 conn(0x7f430806e1e0 0x7f4308153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 >> v1:172.21.15.40:6789/0 conn(0x7f430802c900 legacy=0x7f4308141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43080b7290 con 0x7f43080b5e70 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f42f8001800 con 0x7f43080b5e70 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 >> v1:172.21.15.204:6789/0 conn(0x7f43080b5e70 legacy=0x7f430806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 0x7f42f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4005240 0x7f42f400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b7600 con 0x7f42f4005240 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42f400c1a0 con 0x7f42f4004810 2024-04-02T16:11:07.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f42f400c3a0 con 0x7f42f4003a10 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 0x7f42f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430dbe1640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430ebe3640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4005240 0x7f42f400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 -- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4004810 msgr2=0x7f42f4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42f4004810 0x7f42f4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 -- 172.21.15.204:0/593005984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4005240 msgr2=0x7f42f400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4005240 0x7f42f400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 -- 172.21.15.204:0/593005984 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f4003530 con 0x7f42f4003a10 2024-04-02T16:11:07.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430ebe3640 1 -- 172.21.15.204:0/593005984 reap_dead start 2024-04-02T16:11:07.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430e3e2640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 0x7f42f4003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4300001780 tx=0x7f430001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f80eab5338c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43000013a0 con 0x7f42f4003a10 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42f8007e50 con 0x7f42f4003a10 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.801+0000 7f430d3e0640 1 -- 172.21.15.204:0/593005984 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43000052d0 con 0x7f42f4003a10 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 msgr2=0x7f42f4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 0x7f42f4003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4300001780 tx=0x7f430001c340 comp rx=0 tx=0).stop 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/593005984 shutdown_connections 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 --2- 172.21.15.204:0/593005984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42f4003a10 0x7f42f4003e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/593005984 >> 172.21.15.204:0/593005984 conn(0x7f43080bb3e0 msgr2=0x7f43080bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/593005984 shutdown_connections 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/593005984 wait complete. 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 Processor -- start 2024-04-02T16:11:07.810 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- start start 2024-04-02T16:11:07.812 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4003a10 0x7f4308135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.812 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43081358a0 0x7f43081dd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b6460 con 0x7f42f4003a10 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b62e0 con 0x7f43081358a0 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f43080b65e0 con 0x7f43080b9b70 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58648/0 (socket says 172.21.15.204:58648) 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 -- 172.21.15.204:0/2775073240 learned_addr learned my addr 172.21.15.204:0/2775073240 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430dbe1640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43081358a0 0x7f43081dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430e3e2640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4003a10 0x7f4308135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 -- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43081358a0 msgr2=0x7f43081dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43081358a0 0x7f43081dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 -- 172.21.15.204:0/2775073240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4003a10 msgr2=0x7f4308135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.813 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4003a10 0x7f4308135360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 -- 172.21.15.204:0/2775073240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4308136940 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f430ebe3640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f430806cd10 tx=0x7f43040115a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304011e40 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430401b070 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43080ba090 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43040039a0 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43080ba650 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4304004030 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.805+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f43040169b0 con 0x7f43080b9b70 2024-04-02T16:11:07.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.809+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4308068d00 con 0x7f43080b9b70 2024-04-02T16:11:07.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.817+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4304016470 con 0x7f43080b9b70 2024-04-02T16:11:07.870 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:07.865+0000 7f75cedb8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T16:11:07.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.885+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec", "id": 0} v 0) -- 0x7f43081dd270 con 0x7f43080b9b70 2024-04-02T16:11:07.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f42feffd640 1 -- 172.21.15.204:0/2775073240 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4304016bb0 con 0x7f43080b9b70 2024-04-02T16:11:07.898 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 msgr2=0x7f43081dda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f430806cd10 tx=0x7f43040115a0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 shutdown_connections 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43080b9b70 0x7f43081dda70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43081358a0 0x7f43081dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 --2- 172.21.15.204:0/2775073240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42f4003a10 0x7f4308135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:07.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 >> 172.21.15.204:0/2775073240 conn(0x7f43080bb3e0 msgr2=0x7f4308072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:07.900 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 shutdown_connections 2024-04-02T16:11:07.900 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:07.893+0000 7f4310646640 1 -- 172.21.15.204:0/2775073240 wait complete. 2024-04-02T16:11:07.907 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:07.901+0000 7f3a5c26d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T16:11:07.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 2f24db8f-47da-4934-bd1d-8841a29c88ee 1 2024-04-02T16:11:08.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 Processor -- start 2024-04-02T16:11:08.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 -- start start 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa56c13e2a0 0x7fa56c13e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c13ec60 con 0x7fa56c144920 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c06d010 con 0x7fa56c13e2a0 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa57203e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c06c690 con 0x7fa56c06d8f0 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa563fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa56c144920 0x7fa56c144d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43300/0 (socket says 172.21.15.204:43300) 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58660/0 (socket says 172.21.15.204:58660) 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.025+0000 7fa563fff640 1 -- 172.21.15.204:0/3749376705 learned_addr learned my addr 172.21.15.204:0/3749376705 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56b7fe640 1 -- 172.21.15.204:0/3749376705 >> v2:172.21.15.40:3300/0 conn(0x7fa56c13e2a0 msgr2=0x7fa56c13e720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:11:08.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56b7fe640 1 --2- 172.21.15.204:0/3749376705 >> v2:172.21.15.40:3300/0 conn(0x7fa56c13e2a0 0x7fa56c13e720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56b7fe640 1 -- 172.21.15.204:0/3749376705 >> v1:172.21.15.204:6789/0 conn(0x7fa56c144920 legacy=0x7fa56c144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56b7fe640 1 -- 172.21.15.204:0/3749376705 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0733f0 con 0x7fa56c06d8f0 2024-04-02T16:11:08.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56b7fe640 1 --2- 172.21.15.204:0/3749376705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097c0 tx=0x7fa55c01bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c233dda92d6ee58b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56affd640 1 -- 172.21.15.204:0/3749376705 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0aa640 con 0x7fa56c06d8f0 2024-04-02T16:11:08.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56affd640 1 -- 172.21.15.204:0/3749376705 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c0af070 con 0x7fa56c06d8f0 2024-04-02T16:11:08.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa56affd640 1 -- 172.21.15.204:0/3749376705 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0aaa00 con 0x7fa56c06d8f0 2024-04-02T16:11:08.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 -- 172.21.15.204:0/3749376705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 msgr2=0x7fa56c06dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 --2- 172.21.15.204:0/3749376705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097c0 tx=0x7fa55c01bfa0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 -- 172.21.15.204:0/3749376705 shutdown_connections 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 --2- 172.21.15.204:0/3749376705 >> v2:172.21.15.40:3300/0 conn(0x7fa56c13e2a0 0x7fa56c13e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 --2- 172.21.15.204:0/3749376705 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c06d8f0 0x7fa56c06dcd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 -- 172.21.15.204:0/3749376705 >> 172.21.15.204:0/3749376705 conn(0x7fa56c0b7d10 msgr2=0x7fa56c0b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 -- 172.21.15.204:0/3749376705 shutdown_connections 2024-04-02T16:11:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.029+0000 7fa57203e640 1 -- 172.21.15.204:0/3749376705 wait complete. 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 Processor -- start 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- start start 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c13e2a0 0x7fa56c0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 0x7fa56c141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c152160 con 0x7fa56c13e2a0 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c151fe0 con 0x7fa56c06d8f0 2024-04-02T16:11:08.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c1522e0 con 0x7fa56c144920 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46350/0 (socket says 172.21.15.204:46350) 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56b7fe640 1 -- 172.21.15.204:0/2026770021 learned_addr learned my addr 172.21.15.204:0/2026770021 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 0x7fa56c141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa563fff640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c13e2a0 0x7fa56c0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 -- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 msgr2=0x7fa56c13ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 -- 172.21.15.204:0/2026770021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c13e2a0 msgr2=0x7fa56c0b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c13e2a0 0x7fa56c0b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 -- 172.21.15.204:0/2026770021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c06c890 con 0x7fa56c144920 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56b7fe640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa56bfff640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 0x7fa56c141450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa56400be20 tx=0x7fa56400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564019070 con 0x7fa56c144920 2024-04-02T16:11:08.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564014440 con 0x7fa56c144920 2024-04-02T16:11:08.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564014870 con 0x7fa56c144920 2024-04-02T16:11:08.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c1e4a90 con 0x7fa56c144920 2024-04-02T16:11:08.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c13d760 con 0x7fa56c144920 2024-04-02T16:11:08.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa56401c070 con 0x7fa56c144920 2024-04-02T16:11:08.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa56401d3b0 con 0x7fa56c144920 2024-04-02T16:11:08.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.033+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c068d00 con 0x7fa56c144920 2024-04-02T16:11:08.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.037+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa564023070 con 0x7fa56c144920 2024-04-02T16:11:08.069 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.065+0000 7f75cedb8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T16:11:08.107 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.101+0000 7f3a5c26d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T16:11:08.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.105+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2f24db8f-47da-4934-bd1d-8841a29c88ee", "id": 1} v 0) -- 0x7fa56c0b0590 con 0x7fa56c144920 2024-04-02T16:11:08.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa568ff9640 1 -- 172.21.15.204:0/2026770021 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f24db8f-47da-4934-bd1d-8841a29c88ee", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa56401e590 con 0x7fa56c144920 2024-04-02T16:11:08.122 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-04-02T16:11:08.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 msgr2=0x7fa56c141450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 0x7fa56c141450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa56400be20 tx=0x7fa56400bf10 comp rx=0 tx=0).stop 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 shutdown_connections 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c144920 0x7fa56c141450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa56c13e2a0 0x7fa56c0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 --2- 172.21.15.204:0/2026770021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c06d8f0 0x7fa56c13ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 >> 172.21.15.204:0/2026770021 conn(0x7fa56c0b7d10 msgr2=0x7fa56c0b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 shutdown_connections 2024-04-02T16:11:08.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.117+0000 7fa57203e640 1 -- 172.21.15.204:0/2026770021 wait complete. 2024-04-02T16:11:08.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf 2 2024-04-02T16:11:08.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 Processor -- start 2024-04-02T16:11:08.253 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 -- start start 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 0x7f83c0151710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c00b8140 con 0x7f83c002c460 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c006def0 con 0x7f83c00b5170 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c6279640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83c00b4ce0 con 0x7f83c00b7cd0 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83bf7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f83c00b5170 0x7f83c014dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43314/0 (socket says 172.21.15.204:43314) 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c4816640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 0x7f83c0151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83bf7fe640 1 -- 172.21.15.204:0/3841251199 learned_addr learned my addr 172.21.15.204:0/3841251199 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c4816640 1 -- 172.21.15.204:0/3841251199 >> v1:172.21.15.204:6789/0 conn(0x7f83c00b5170 legacy=0x7f83c014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.254 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c4816640 1 -- 172.21.15.204:0/3841251199 >> v1:172.21.15.40:6789/0 conn(0x7f83c002c460 legacy=0x7f83c013f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:08.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83beffd640 1 -- 172.21.15.204:0/3841251199 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 637512786 0 0) 0x7f83c006def0 con 0x7f83c00b5170 2024-04-02T16:11:08.255 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.249+0000 7f83c4816640 1 -- 172.21.15.204:0/3841251199 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c006e1d0 con 0x7f83c00b7cd0 2024-04-02T16:11:08.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c4816640 1 --2- 172.21.15.204:0/3841251199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 0x7f83c0151710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fbd34c39ee5076 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83beffd640 1 -- 172.21.15.204:0/3841251199 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b80b2070 con 0x7f83c00b7cd0 2024-04-02T16:11:08.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83beffd640 1 -- 172.21.15.204:0/3841251199 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b80a5920 con 0x7f83c00b7cd0 2024-04-02T16:11:08.256 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83beffd640 1 -- 172.21.15.204:0/3841251199 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b80a5cf0 con 0x7f83c00b7cd0 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/3841251199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 msgr2=0x7f83c0151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 --2- 172.21.15.204:0/3841251199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 0x7f83c0151710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b801c950 comp rx=0 tx=0).stop 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/3841251199 shutdown_connections 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 --2- 172.21.15.204:0/3841251199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c00b7cd0 0x7f83c0151710 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/3841251199 >> 172.21.15.204:0/3841251199 conn(0x7f83c00b5fb0 msgr2=0x7f83c00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.257 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/3841251199 shutdown_connections 2024-04-02T16:11:08.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/3841251199 wait complete. 2024-04-02T16:11:08.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 Processor -- start 2024-04-02T16:11:08.258 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- start start 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 0x7f83c0072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b7cd0 0x7f83c0070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83c00b7430 con 0x7f83c00b7cd0 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83c00b72b0 con 0x7f83c00b5170 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83c00b5c30 con 0x7f83c002c460 2024-04-02T16:11:08.260 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 0x7f83c0072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58684/0 (socket says 172.21.15.204:58684) 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 -- 172.21.15.204:0/1782478887 learned_addr learned my addr 172.21.15.204:0/1782478887 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 -- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 msgr2=0x7f83c0072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 0x7f83c0072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 -- 172.21.15.204:0/1782478887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b7cd0 msgr2=0x7f83c0070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b7cd0 0x7f83c0070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 -- 172.21.15.204:0/1782478887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c0071e10 con 0x7f83c002c460 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bf7fe640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 0x7f83c0072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bffff640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c450 tx=0x7f83b400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4012830 con 0x7f83c002c460 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4017070 con 0x7f83c002c460 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c00b0d30 con 0x7f83c002c460 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4012c30 con 0x7f83c002c460 2024-04-02T16:11:08.261 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c006db60 con 0x7f83c002c460 2024-04-02T16:11:08.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83b401c460 con 0x7f83c002c460 2024-04-02T16:11:08.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.253+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f83b401bb50 con 0x7f83c002c460 2024-04-02T16:11:08.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.257+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c0068d00 con 0x7f83c002c460 2024-04-02T16:11:08.262 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.257+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f83b401bdf0 con 0x7f83c002c460 2024-04-02T16:11:08.330 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.325+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf", "id": 2} v 0) -- 0x7f83c00b8df0 con 0x7f83c002c460 2024-04-02T16:11:08.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83bcff9640 1 -- 172.21.15.204:0/1782478887 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f83b401b730 con 0x7f83c002c460 2024-04-02T16:11:08.338 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 msgr2=0x7f83c013afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c450 tx=0x7f83b400c910 comp rx=0 tx=0).stop 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 shutdown_connections 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83c00b7cd0 0x7f83c0070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83c00b5170 0x7f83c0072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 --2- 172.21.15.204:0/1782478887 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83c002c460 0x7f83c013afb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 >> 172.21.15.204:0/1782478887 conn(0x7f83c00b5fb0 msgr2=0x7f83c00672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 shutdown_connections 2024-04-02T16:11:08.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.333+0000 7f83c6279640 1 -- 172.21.15.204:0/1782478887 wait complete. 2024-04-02T16:11:08.354 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new e06f6d74-bb47-480c-9d67-0b6a76e482b7 3 2024-04-02T16:11:08.471 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.465+0000 7f1e4f5cf640 1 Processor -- start 2024-04-02T16:11:08.471 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.465+0000 7f1e4f5cf640 1 -- start start 2024-04-02T16:11:08.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e4806d8f0 0x7f1e4806dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e4813e2a0 0x7f1e4813e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4806d010 con 0x7f1e48144920 2024-04-02T16:11:08.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4813ec90 con 0x7f1e4806d8f0 2024-04-02T16:11:08.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4806c690 con 0x7f1e4813e2a0 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e4813e2a0 0x7f1e4813e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1e4813e2a0 0x7f1e4813e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46360/0 (socket says 172.21.15.204:46360) 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 -- 172.21.15.204:0/3976816221 learned_addr learned my addr 172.21.15.204:0/3976816221 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4d36b640 1 --2- 172.21.15.204:0/3976816221 >> v2:172.21.15.40:3300/0 conn(0x7f1e4806d8f0 0x7f1e4806dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 -- 172.21.15.204:0/3976816221 >> v2:172.21.15.40:3300/0 conn(0x7f1e4806d8f0 msgr2=0x7f1e4806dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 --2- 172.21.15.204:0/3976816221 >> v2:172.21.15.40:3300/0 conn(0x7f1e4806d8f0 0x7f1e4806dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 -- 172.21.15.204:0/3976816221 >> v1:172.21.15.40:6789/0 conn(0x7f1e48144920 legacy=0x7f1e48144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:08.474 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 -- 172.21.15.204:0/3976816221 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e480733f0 con 0x7f1e4813e2a0 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4db6c640 1 --2- 172.21.15.204:0/3976816221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e4813e2a0 0x7f1e4813e750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806e240 tx=0x7f1e4001eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb3fc8ada4585b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e3ffff640 1 -- 172.21.15.204:0/3976816221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000b2d0 con 0x7f1e4813e2a0 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e3ffff640 1 -- 172.21.15.204:0/3976816221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4000b470 con 0x7f1e4813e2a0 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e3ffff640 1 -- 172.21.15.204:0/3976816221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e400a7910 con 0x7f1e4813e2a0 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3976816221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e4813e2a0 msgr2=0x7f1e4813e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3976816221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e4813e2a0 0x7f1e4813e750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1e4806e240 tx=0x7f1e4001eb20 comp rx=0 tx=0).stop 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3976816221 shutdown_connections 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3976816221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e4813e2a0 0x7f1e4813e750 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3976816221 >> v2:172.21.15.40:3300/0 conn(0x7f1e4806d8f0 0x7f1e4806dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3976816221 >> 172.21.15.204:0/3976816221 conn(0x7f1e480b7d10 msgr2=0x7f1e480b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.476 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3976816221 shutdown_connections 2024-04-02T16:11:08.477 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.469+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3976816221 wait complete. 2024-04-02T16:11:08.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 Processor -- start 2024-04-02T16:11:08.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- start start 2024-04-02T16:11:08.479 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 0x7f1e481dc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e48144920 0x7f1e48133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4806c120 con 0x7f1e4806d8f0 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4806bfa0 con 0x7f1e48144920 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4806c2a0 con 0x7f1e4813e2a0 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4db6c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e48144920 0x7f1e48133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58692/0 (socket says 172.21.15.204:58692) 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 -- 172.21.15.204:0/3007905095 learned_addr learned my addr 172.21.15.204:0/3007905095 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4d36b640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 0x7f1e481dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 -- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e48144920 msgr2=0x7f1e48133f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e48144920 0x7f1e48133f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 -- 172.21.15.204:0/3007905095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 msgr2=0x7f1e481dc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.480 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 0x7f1e481dc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 -- 172.21.15.204:0/3007905095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481dcb60 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4d36b640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 0x7f1e481dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4cb6a640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800b410 tx=0x7f1e3800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38015030 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e38004020 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e481dbe20 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38004410 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481368a0 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1e38018070 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1e3801a410 con 0x7f1e4813e2a0 2024-04-02T16:11:08.481 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.473+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e481dbc10 con 0x7f1e4813e2a0 2024-04-02T16:11:08.482 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.477+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1e38003a10 con 0x7f1e4813e2a0 2024-04-02T16:11:08.549 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.545+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e06f6d74-bb47-480c-9d67-0b6a76e482b7", "id": 3} v 0) -- 0x7f1e48068d00 con 0x7f1e4813e2a0 2024-04-02T16:11:08.556 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e3dffb640 1 -- 172.21.15.204:0/3007905095 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e06f6d74-bb47-480c-9d67-0b6a76e482b7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1e38003c90 con 0x7f1e4813e2a0 2024-04-02T16:11:08.556 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 msgr2=0x7f1e48139c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800b410 tx=0x7f1e3800b8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 shutdown_connections 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1e4813e2a0 0x7f1e48139c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e48144920 0x7f1e48133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 --2- 172.21.15.204:0/3007905095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e4806d8f0 0x7f1e481dc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 >> 172.21.15.204:0/3007905095 conn(0x7f1e480b7d10 msgr2=0x7f1e480b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 shutdown_connections 2024-04-02T16:11:08.558 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.553+0000 7f1e4f5cf640 1 -- 172.21.15.204:0/3007905095 wait complete. 2024-04-02T16:11:08.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 3e262364-3787-4159-a389-beb019a2305d 4 2024-04-02T16:11:08.671 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.665+0000 7f75cedb8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T16:11:08.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 Processor -- start 2024-04-02T16:11:08.687 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 -- start start 2024-04-02T16:11:08.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 0x7f36300b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 0x7f3630141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36300b7130 con 0x7f36300728c0 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36300b56e0 con 0x7f36300b6d50 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f3636362640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3630070a50 con 0x7f3630141650 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f36348ff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f36300728c0 0x7f363006b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43326/0 (socket says 172.21.15.204:43326) 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f36348ff640 1 -- 172.21.15.204:0/3025997991 learned_addr learned my addr 172.21.15.204:0/3025997991 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f362f7fe640 1 --2- 172.21.15.204:0/3025997991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 0x7f3630141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f362effd640 1 -- 172.21.15.204:0/3025997991 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020510274 0 0) 0x7f36300b7130 con 0x7f36300728c0 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.681+0000 7f362effd640 1 -- 172.21.15.204:0/3025997991 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3618003530 con 0x7f36300728c0 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362ffff640 1 --2- 172.21.15.204:0/3025997991 >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 0x7f36300b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362f7fe640 1 -- 172.21.15.204:0/3025997991 >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 msgr2=0x7f36300b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362f7fe640 1 --2- 172.21.15.204:0/3025997991 >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 0x7f36300b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362f7fe640 1 -- 172.21.15.204:0/3025997991 >> v1:172.21.15.204:6789/0 conn(0x7f36300728c0 legacy=0x7f363006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362f7fe640 1 -- 172.21.15.204:0/3025997991 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3630070ca0 con 0x7f3630141650 2024-04-02T16:11:08.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362ffff640 1 --2- 172.21.15.204:0/3025997991 >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 0x7f36300b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:08.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362f7fe640 1 --2- 172.21.15.204:0/3025997991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 0x7f3630141ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36200097c0 tx=0x7f362001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2100bf6f6c2dc83c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362effd640 1 -- 172.21.15.204:0/3025997991 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200b0070 con 0x7f3630141650 2024-04-02T16:11:08.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362effd640 1 -- 172.21.15.204:0/3025997991 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36200ab600 con 0x7f3630141650 2024-04-02T16:11:08.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f362effd640 1 -- 172.21.15.204:0/3025997991 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200ab9d0 con 0x7f3630141650 2024-04-02T16:11:08.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 -- 172.21.15.204:0/3025997991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 msgr2=0x7f3630141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 --2- 172.21.15.204:0/3025997991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 0x7f3630141ab0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36200097c0 tx=0x7f362001c410 comp rx=0 tx=0).stop 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 -- 172.21.15.204:0/3025997991 shutdown_connections 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 --2- 172.21.15.204:0/3025997991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3630141650 0x7f3630141ab0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 --2- 172.21.15.204:0/3025997991 >> v2:172.21.15.40:3300/0 conn(0x7f36300b6d50 0x7f36300b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 -- 172.21.15.204:0/3025997991 >> 172.21.15.204:0/3025997991 conn(0x7f363006c860 msgr2=0x7f363006cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 -- 172.21.15.204:0/3025997991 shutdown_connections 2024-04-02T16:11:08.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.685+0000 7f3636362640 1 -- 172.21.15.204:0/3025997991 wait complete. 2024-04-02T16:11:08.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 Processor -- start 2024-04-02T16:11:08.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- start start 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36300728c0 0x7f36300ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3630141650 0x7f36301461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36301551c0 con 0x7f36300728c0 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3630155040 con 0x7f3630141650 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3630155340 con 0x7f36300b6d50 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58720/0 (socket says 172.21.15.204:58720) 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 -- 172.21.15.204:0/4093609950 learned_addr learned my addr 172.21.15.204:0/4093609950 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f36348ff640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3630141650 0x7f36301461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 -- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3630141650 msgr2=0x7f36301461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3630141650 0x7f36301461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 -- 172.21.15.204:0/4093609950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36300728c0 msgr2=0x7f36300ae4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36300728c0 0x7f36300ae4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 -- 172.21.15.204:0/4093609950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36300ba980 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362f7fe640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f362001a040 tx=0x7f36200191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200ba070 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36200b43f0 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200b0040 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36300aff00 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36300af120 con 0x7f36300b6d50 2024-04-02T16:11:08.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36200b47c0 con 0x7f36300b6d50 2024-04-02T16:11:08.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f36200aa8b0 con 0x7f36300b6d50 2024-04-02T16:11:08.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.689+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3630068d00 con 0x7f36300b6d50 2024-04-02T16:11:08.697 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.693+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f36200c3070 con 0x7f36300b6d50 2024-04-02T16:11:08.701 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.697+0000 7f3a5c26d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T16:11:08.727 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.721+0000 7f75cedb8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T16:11:08.757 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.753+0000 7f3a5c26d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T16:11:08.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.761+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3e262364-3787-4159-a389-beb019a2305d", "id": 4} v 0) -- 0x7f3630070830 con 0x7f36300b6d50 2024-04-02T16:11:08.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.765+0000 7f362cff9640 1 -- 172.21.15.204:0/4093609950 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e262364-3787-4159-a389-beb019a2305d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3620016090 con 0x7f36300b6d50 2024-04-02T16:11:08.771 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 msgr2=0x7f3630145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f362001a040 tx=0x7f36200191b0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 shutdown_connections 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36300b6d50 0x7f3630145590 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f362001a040 tx=0x7f36200191b0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3630141650 0x7f36301461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 --2- 172.21.15.204:0/4093609950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36300728c0 0x7f36300ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 >> 172.21.15.204:0/4093609950 conn(0x7f363006c860 msgr2=0x7f363013e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 shutdown_connections 2024-04-02T16:11:08.773 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.769+0000 7f3636362640 1 -- 172.21.15.204:0/4093609950 wait complete. 2024-04-02T16:11:08.787 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.781+0000 7f75cedb8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T16:11:08.793 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 8af58e19-9afb-40f2-a32a-9b94817636d3 5 2024-04-02T16:11:08.812 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.809+0000 7f3a5c26d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T16:11:08.869 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.865+0000 7f75cedb8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T16:11:08.892 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.889+0000 7f3a5c26d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T16:11:08.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 Processor -- start 2024-04-02T16:11:08.906 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 -- start start 2024-04-02T16:11:08.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe3100b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe31006da10 0x7fe31006de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe31006e3b0 con 0x7fe3100b59b0 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe3100b6ea0 con 0x7fe31006da10 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe315836640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe3100b7460 con 0x7fe31002c860 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe3100b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe31006da10 0x7fe31006de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe31006da10 0x7fe31006de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46386/0 (socket says 172.21.15.204:46386) 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe3100b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58732/0 (socket says 172.21.15.204:58732) 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 -- 172.21.15.204:0/3272310329 learned_addr learned my addr 172.21.15.204:0/3272310329 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 -- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 msgr2=0x7fe3100b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 --2- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe3100b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 -- 172.21.15.204:0/3272310329 >> v1:172.21.15.40:6789/0 conn(0x7fe3100b59b0 legacy=0x7fe310141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.908 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.901+0000 7fe30f7fe640 1 -- 172.21.15.204:0/3272310329 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31002cc40 con 0x7fe31006da10 2024-04-02T16:11:08.909 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe30f7fe640 1 --2- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe31006de70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b7020 tx=0x7fe30801c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ba19a540ec29e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe30dffb640 1 -- 172.21.15.204:0/3272310329 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080a5680 con 0x7fe31006da10 2024-04-02T16:11:08.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe30dffb640 1 -- 172.21.15.204:0/3272310329 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3080a5820 con 0x7fe31006da10 2024-04-02T16:11:08.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe30dffb640 1 -- 172.21.15.204:0/3272310329 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080a5bf0 con 0x7fe31006da10 2024-04-02T16:11:08.910 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 msgr2=0x7fe31006de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 --2- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe31006de70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b7020 tx=0x7fe30801c8f0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- 172.21.15.204:0/3272310329 shutdown_connections 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 --2- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe31006de70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 --2- 172.21.15.204:0/3272310329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe3100b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- 172.21.15.204:0/3272310329 >> 172.21.15.204:0/3272310329 conn(0x7fe3100bb3e0 msgr2=0x7fe3100bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- 172.21.15.204:0/3272310329 shutdown_connections 2024-04-02T16:11:08.911 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- 172.21.15.204:0/3272310329 wait complete. 2024-04-02T16:11:08.912 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 Processor -- start 2024-04-02T16:11:08.912 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.905+0000 7fe315836640 1 -- start start 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe310142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 0x7fe310137240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe310154b70 con 0x7fe3100b59b0 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe3101549f0 con 0x7fe31006da10 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe310154cf0 con 0x7fe31002c860 2024-04-02T16:11:08.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58736/0 (socket says 172.21.15.204:58736) 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 -- 172.21.15.204:0/856666273 learned_addr learned my addr 172.21.15.204:0/856666273 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe310142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30f7fe640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 0x7fe310137240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 -- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 msgr2=0x7fe310142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe310142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 -- 172.21.15.204:0/856666273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 msgr2=0x7fe310137240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 0x7fe310137240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 -- 172.21.15.204:0/856666273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3101432c0 con 0x7fe31002c860 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30f7fe640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 0x7fe310137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30e7fc640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe310142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe30effd640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe30000e960 tx=0x7fe30000ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300017070 con 0x7fe31002c860 2024-04-02T16:11:08.915 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30000ca00 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30000cdb0 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe310143540 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31013a0d0 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe30001a070 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe300006610 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.909+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3101427f0 con 0x7fe31002c860 2024-04-02T16:11:08.916 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.913+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe3101432c0 con 0x7fe31002c860 2024-04-02T16:11:08.923 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.917+0000 7f75cedb8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T16:11:08.946 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:08.941+0000 7f3a5c26d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T16:11:08.979 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.973+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8af58e19-9afb-40f2-a32a-9b94817636d3", "id": 5} v 0) -- 0x7fe310068d00 con 0x7fe31002c860 2024-04-02T16:11:08.986 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe2f7fff640 1 -- 172.21.15.204:0/856666273 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8af58e19-9afb-40f2-a32a-9b94817636d3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe3000059a0 con 0x7fe31002c860 2024-04-02T16:11:08.987 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 msgr2=0x7fe310147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe30000e960 tx=0x7fe30000ee20 comp rx=0 tx=0).stop 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 shutdown_connections 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100b59b0 0x7fe310137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe31006da10 0x7fe310142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 --2- 172.21.15.204:0/856666273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe31002c860 0x7fe310147aa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 >> 172.21.15.204:0/856666273 conn(0x7fe3100bb3e0 msgr2=0x7fe3100b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 shutdown_connections 2024-04-02T16:11:08.988 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:08.981+0000 7fe315836640 1 -- 172.21.15.204:0/856666273 wait complete. 2024-04-02T16:11:09.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 00d9c46d-e5b7-4097-85fe-8434592d792b 6 2024-04-02T16:11:09.003 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:08.997+0000 7f75cedb8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T16:11:09.027 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:09.021+0000 7f3a5c26d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T16:11:09.057 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:09.053+0000 7f75cedb8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T16:11:09.082 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:09.077+0000 7f3a5c26d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T16:11:09.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 Processor -- start 2024-04-02T16:11:09.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 -- start start 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8c700b59b0 0x7f8c700b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7002cc40 con 0x7f8c7002c860 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c700b70f0 con 0x7f8c7006da10 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c74cb1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8c700b6ea0 con 0x7f8c700b59b0 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c6dd9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8c700b59b0 0x7f8c700b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c6ed9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8c7006da10 0x7f8c7006dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43328/0 (socket says 172.21.15.204:43328) 2024-04-02T16:11:09.120 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.113+0000 7f8c6ed9d640 1 -- 172.21.15.204:0/441615841 learned_addr learned my addr 172.21.15.204:0/441615841 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-04-02T16:11:09.121 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6dd9b640 1 -- 172.21.15.204:0/441615841 >> v1:172.21.15.204:6789/0 conn(0x7f8c7006da10 legacy=0x7f8c7006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.121 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6dd9b640 1 -- 172.21.15.204:0/441615841 >> v1:172.21.15.40:6789/0 conn(0x7f8c7002c860 legacy=0x7f8c70141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:09.121 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6dd9b640 1 -- 172.21.15.204:0/441615841 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70141e20 con 0x7f8c700b59b0 2024-04-02T16:11:09.121 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6d59a640 1 -- 172.21.15.204:0/441615841 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3581106928 0 0) 0x7f8c700b70f0 con 0x7f8c7006da10 2024-04-02T16:11:09.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6dd9b640 1 --2- 172.21.15.204:0/441615841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c700b59b0 0x7f8c700b6350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c580097a0 tx=0x7f8c5801c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb76c767786267e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6d59a640 1 -- 172.21.15.204:0/441615841 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c580b0070 con 0x7f8c700b59b0 2024-04-02T16:11:09.122 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6d59a640 1 -- 172.21.15.204:0/441615841 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c580ab600 con 0x7f8c700b59b0 2024-04-02T16:11:09.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c6d59a640 1 -- 172.21.15.204:0/441615841 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c580ab9d0 con 0x7f8c700b59b0 2024-04-02T16:11:09.123 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 -- 172.21.15.204:0/441615841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c700b59b0 msgr2=0x7f8c700b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/441615841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c700b59b0 0x7f8c700b6350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c580097a0 tx=0x7f8c5801c320 comp rx=0 tx=0).stop 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 -- 172.21.15.204:0/441615841 shutdown_connections 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/441615841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c700b59b0 0x7f8c700b6350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 -- 172.21.15.204:0/441615841 >> 172.21.15.204:0/441615841 conn(0x7f8c700bb3e0 msgr2=0x7f8c700bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 -- 172.21.15.204:0/441615841 shutdown_connections 2024-04-02T16:11:09.124 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.117+0000 7f8c74cb1640 1 -- 172.21.15.204:0/441615841 wait complete. 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 Processor -- start 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- start start 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 0x7f8c700b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 0x7f8c700b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70155b80 con 0x7f8c7006da10 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70155a00 con 0x7f8c7002c860 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8c70155d00 con 0x7f8c700b59b0 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 0x7f8c700b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58746/0 (socket says 172.21.15.204:58746) 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6ed9d640 1 -- 172.21.15.204:0/578064694 learned_addr learned my addr 172.21.15.204:0/578064694 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6dd9b640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 0x7f8c700b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 -- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 msgr2=0x7f8c70143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 -- 172.21.15.204:0/578064694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 msgr2=0x7f8c700b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 0x7f8c700b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 -- 172.21.15.204:0/578064694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c700b2350 con 0x7f8c7002c860 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6dd9b640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 0x7f8c700b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6ed9d640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c6e59c640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 0x7f8c700b3380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b440 tx=0x7f8c6800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68011830 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68016070 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c700b4830 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68011c10 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c70143c40 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c68004020 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f8c6801a9b0 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.121+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c70068d00 con 0x7f8c7002c860 2024-04-02T16:11:09.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.125+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8c68020020 con 0x7f8c7002c860 2024-04-02T16:11:09.198 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.193+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00d9c46d-e5b7-4097-85fe-8434592d792b", "id": 6} v 0) -- 0x7f8c700b26e0 con 0x7f8c7002c860 2024-04-02T16:11:09.204 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.197+0000 7f8c66ffd640 1 -- 172.21.15.204:0/578064694 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00d9c46d-e5b7-4097-85fe-8434592d792b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8c68011db0 con 0x7f8c7002c860 2024-04-02T16:11:09.204 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 msgr2=0x7f8c700b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 0x7f8c700b3380 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800b440 tx=0x7f8c6800b900 comp rx=0 tx=0).stop 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 shutdown_connections 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c700b59b0 0x7f8c70143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7006da10 0x7f8c700b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 --2- 172.21.15.204:0/578064694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c7002c860 0x7f8c700b3380 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 >> 172.21.15.204:0/578064694 conn(0x7f8c700bb3e0 msgr2=0x7f8c700b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 shutdown_connections 2024-04-02T16:11:09.206 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.201+0000 7f8c74cb1640 1 -- 172.21.15.204:0/578064694 wait complete. 2024-04-02T16:11:09.220 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:09.217+0000 7f75cedb8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T16:11:09.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 242ea867-d592-4b2b-a673-bfd5739b9791 7 2024-04-02T16:11:09.239 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:09.233+0000 7f3a5c26d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T16:11:09.285 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:09.281+0000 7f75cedb8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T16:11:09.303 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:09.297+0000 7f3a5c26d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T16:11:09.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 Processor -- start 2024-04-02T16:11:09.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 -- start start 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0b5a50 con 0x7f5ebc0b7250 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0b31f0 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec40b0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc145c10 con 0x7f5ebc0b4a50 2024-04-02T16:11:09.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec264d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46424/0 (socket says 172.21.15.204:46424) 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec264d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58762/0 (socket says 172.21.15.204:58762) 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec1e4c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5ebc0b7250 0x7f5ebc145500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:43332/0 (socket says 172.21.15.204:43332) 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 -- 172.21.15.204:0/2425132687 learned_addr learned my addr 172.21.15.204:0/2425132687 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 -- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 msgr2=0x7f5ebc0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 -- 172.21.15.204:0/2425132687 >> v1:172.21.15.204:6789/0 conn(0x7f5ebc0b7250 legacy=0x7f5ebc145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 -- 172.21.15.204:0/2425132687 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0b7630 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec264d640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:09.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec164b640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5eac006fd0 tx=0x7f5eac01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae5906b2742c0720 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec0e4a640 1 -- 172.21.15.204:0/2425132687 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0ab680 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec0e4a640 1 -- 172.21.15.204:0/2425132687 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac0b0070 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.333+0000 7f5ec0e4a640 1 -- 172.21.15.204:0/2425132687 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0aba40 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 msgr2=0x7f5ebc02cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5eac006fd0 tx=0x7f5eac01c480 comp rx=0 tx=0).stop 2024-04-02T16:11:09.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2425132687 shutdown_connections 2024-04-02T16:11:09.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc0b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2425132687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc02cb80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2425132687 >> 172.21.15.204:0/2425132687 conn(0x7f5ebc0bb3e0 msgr2=0x7f5ebc0bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2425132687 shutdown_connections 2024-04-02T16:11:09.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2425132687 wait complete. 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 Processor -- start 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- start start 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0b7250 0x7f5ebc13d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0b64c0 con 0x7f5ebc0b7250 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc152620 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc136bb0 con 0x7f5ebc0b4a50 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec164b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec164b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc13cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58774/0 (socket says 172.21.15.204:58774) 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec164b640 1 -- 172.21.15.204:0/2227164778 learned_addr learned my addr 172.21.15.204:0/2227164778 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec264d640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0b7250 0x7f5ebc13d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 -- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 msgr2=0x7f5ebc13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 -- 172.21.15.204:0/2227164778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0b7250 msgr2=0x7f5ebc13d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0b7250 0x7f5ebc13d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 -- 172.21.15.204:0/2227164778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc06e1b0 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec1e4c640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc144720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c390 tx=0x7f5eb800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80074e0 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8007680 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc13c8a0 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004a10 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc06e430 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5eb801a020 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f5eb8012a50 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.337+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc13ca70 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.341+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5eb8021070 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.414 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.409+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "242ea867-d592-4b2b-a673-bfd5739b9791", "id": 7} v 0) -- 0x7f5ebc068d00 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.423 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.417+0000 7f5eb27fc640 1 -- 172.21.15.204:0/2227164778 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "242ea867-d592-4b2b-a673-bfd5739b9791", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f5ebc06e1b0 con 0x7f5ebc02c6f0 2024-04-02T16:11:09.424 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 msgr2=0x7f5ebc144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc144720 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c390 tx=0x7f5eb800c850 comp rx=0 tx=0).stop 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 shutdown_connections 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ebc0b4a50 0x7f5ebc13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ebc0b7250 0x7f5ebc13d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 --2- 172.21.15.204:0/2227164778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ebc02c6f0 0x7f5ebc144720 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 >> 172.21.15.204:0/2227164778 conn(0x7f5ebc0bb3e0 msgr2=0x7f5ebc0ba6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 shutdown_connections 2024-04-02T16:11:09.425 INFO:teuthology.orchestra.run.smithi204.stderr:2024-04-02T16:11:09.421+0000 7f5ec40b0640 1 -- 172.21.15.204:0/2227164778 wait complete. 2024-04-02T16:11:09.439 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-02T16:11:09.440 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-02T16:11:09.442 INFO:tasks.ceph.osd.0:Started 2024-04-02T16:11:09.442 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-02T16:11:09.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-02T16:11:09.444 INFO:tasks.ceph.osd.1:Started 2024-04-02T16:11:09.445 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-02T16:11:09.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-02T16:11:09.447 INFO:tasks.ceph.osd.2:Started 2024-04-02T16:11:09.447 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-02T16:11:09.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-02T16:11:09.449 INFO:tasks.ceph.osd.3:Started 2024-04-02T16:11:09.449 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-02T16:11:09.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-02T16:11:09.451 INFO:tasks.ceph.osd.4:Started 2024-04-02T16:11:09.451 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-02T16:11:09.451 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-02T16:11:09.453 INFO:tasks.ceph.osd.5:Started 2024-04-02T16:11:09.453 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-02T16:11:09.453 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-02T16:11:09.454 INFO:tasks.ceph.osd.6:Started 2024-04-02T16:11:09.455 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-02T16:11:09.455 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-02T16:11:09.456 INFO:tasks.ceph.osd.7:Started 2024-04-02T16:11:09.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:09.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 Processor -- start 2024-04-02T16:11:09.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 -- start start 2024-04-02T16:11:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28180b4bf0 0x7f28180b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b8c80 0x7f28180b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28180b6310 con 0x7f28180b3500 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f28180b55d0 con 0x7f28180b4bf0 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281e791640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f281814b560 con 0x7f28180b8c80 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f2817fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28180b4bf0 0x7f28180b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f280ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f28180b3500 0x7f281814ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38866/0 (socket says 172.21.15.40:38866) 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f280ffff640 1 -- 172.21.15.40:0/3782532786 learned_addr learned my addr 172.21.15.40:0/3782532786 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f281cd2e640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b8c80 0x7f28180b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f2817fff640 1 -- 172.21.15.40:0/3782532786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b8c80 msgr2=0x7f28180b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f2817fff640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b8c80 0x7f28180b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f2817fff640 1 -- 172.21.15.40:0/3782532786 >> v1:172.21.15.40:6789/0 conn(0x7f28180b3500 legacy=0x7f281814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f28177fe640 1 -- 172.21.15.40:0/3782532786 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649329930 0 0) 0x7f28180b6310 con 0x7f28180b3500 2024-04-02T16:11:09.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.573+0000 7f2817fff640 1 -- 172.21.15.40:0/3782532786 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28180b3fa0 con 0x7f28180b4bf0 2024-04-02T16:11:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f2817fff640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b4bf0 0x7f28180b4fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28080097c0 tx=0x7f280801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81726989f7162e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f28177fe640 1 -- 172.21.15.40:0/3782532786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080b0070 con 0x7f28180b4bf0 2024-04-02T16:11:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f28177fe640 1 -- 172.21.15.40:0/3782532786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28080ab600 con 0x7f28180b4bf0 2024-04-02T16:11:09.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f28177fe640 1 -- 172.21.15.40:0/3782532786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080ab9d0 con 0x7f28180b4bf0 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 -- 172.21.15.40:0/3782532786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b4bf0 msgr2=0x7f28180b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b4bf0 0x7f28180b4fd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28080097c0 tx=0x7f280801c380 comp rx=0 tx=0).stop 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 -- 172.21.15.40:0/3782532786 shutdown_connections 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b8c80 0x7f28180b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 --2- 172.21.15.40:0/3782532786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b4bf0 0x7f28180b4fd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 -- 172.21.15.40:0/3782532786 >> 172.21.15.40:0/3782532786 conn(0x7f28180734b0 msgr2=0x7f28180bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 -- 172.21.15.40:0/3782532786 shutdown_connections 2024-04-02T16:11:09.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.577+0000 7f281e791640 1 -- 172.21.15.40:0/3782532786 wait complete. 2024-04-02T16:11:09.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 Processor -- start 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- start start 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b4bf0 0x7f28181e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28180b8c80 0x7f28181e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2818154c10 con 0x7f28180b3500 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2818154d90 con 0x7f28180b8c80 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2818154f10 con 0x7f28180b4bf0 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43486/0 (socket says 172.21.15.40:43486) 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 -- 172.21.15.40:0/2602730219 learned_addr learned my addr 172.21.15.40:0/2602730219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 -- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b4bf0 msgr2=0x7f28181e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b4bf0 0x7f28181e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 -- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28180b8c80 msgr2=0x7f28181e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28180b8c80 0x7f28181e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 -- 172.21.15.40:0/2602730219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281814cd10 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f2817fff640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2808019fb0 tx=0x7f2808019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080b8020 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28080aa7d0 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2818141450 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28080b0040 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2818141170 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f28080bb020 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f28080b4d20 con 0x7f28180b3500 2024-04-02T16:11:09.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.581+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28180685f0 con 0x7f28180b3500 2024-04-02T16:11:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.601+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f28080c2030 con 0x7f28180b3500 2024-04-02T16:11:09.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.697+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28180b8a60 con 0x7f28180b3500 2024-04-02T16:11:09.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.697+0000 7f28157fa640 1 -- 172.21.15.40:0/2602730219 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2808016090 con 0x7f28180b3500 2024-04-02T16:11:09.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:09.703 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:09.416797+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 msgr2=0x7f2818143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2808019fb0 tx=0x7f2808019120 comp rx=0 tx=0).stop 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 shutdown_connections 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28180b4bf0 0x7f28181e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28180b8c80 0x7f28181e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 --2- 172.21.15.40:0/2602730219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28180b3500 0x7f2818143fc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:09.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 >> 172.21.15.40:0/2602730219 conn(0x7f28180734b0 msgr2=0x7f28180bc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:09.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 shutdown_connections 2024-04-02T16:11:09.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:09.701+0000 7f281e791640 1 -- 172.21.15.40:0/2602730219 wait complete. 2024-04-02T16:11:09.719 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-02T16:11:09.719 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-02T16:11:09.861 INFO:tasks.ceph.mgr.x.smithi204.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-02T16:11:09.861 INFO:tasks.ceph.mgr.x.smithi204.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-02T16:11:09.861 INFO:tasks.ceph.mgr.x.smithi204.stderr: from numpy import show_config as show_numpy_config 2024-04-02T16:11:09.921 INFO:tasks.ceph.mgr.y.smithi040.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-02T16:11:09.922 INFO:tasks.ceph.mgr.y.smithi040.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-02T16:11:09.922 INFO:tasks.ceph.mgr.y.smithi040.stderr: from numpy import show_config as show_numpy_config 2024-04-02T16:11:09.935 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:09.929+0000 7f75cedb8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T16:11:09.998 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:09.993+0000 7f3a5c26d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T16:11:10.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-02T16:11:10.049+0000 7f454b53b7c0 -1 Falling back to public interface 2024-04-02T16:11:10.055 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-02T16:11:10.049+0000 7f12f52617c0 -1 Falling back to public interface 2024-04-02T16:11:10.055 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f37284827c0 -1 Falling back to public interface 2024-04-02T16:11:10.056 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f892745c7c0 -1 Falling back to public interface 2024-04-02T16:11:10.056 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f01ec8327c0 -1 Falling back to public interface 2024-04-02T16:11:10.056 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f11fd54c7c0 -1 Falling back to public interface 2024-04-02T16:11:10.057 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f892641c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-02T16:11:10.057 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-02T16:11:10.049+0000 7f3727442640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-02T16:11:10.068 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-02T16:11:10.065+0000 7fc54d1147c0 -1 Falling back to public interface 2024-04-02T16:11:10.084 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-02T16:11:10.081+0000 7f27196df7c0 -1 Falling back to public interface 2024-04-02T16:11:10.120 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:10.113+0000 7f75cedb8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:11:10.188 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:10.185+0000 7f3a5c26d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:11:10.285 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:10.281+0000 7f75cedb8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T16:11:10.355 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:10.349+0000 7f3a5c26d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T16:11:10.355 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:10.349+0000 7f75cedb8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T16:11:10.426 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:10.421+0000 7f3a5c26d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T16:11:10.501 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:10.497+0000 7f75cedb8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:11:10.525 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:10.576 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:10.573+0000 7f3a5c26d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T16:11:10.647 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.641+0000 7f14ab35c640 1 Processor -- start 2024-04-02T16:11:10.647 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.641+0000 7f14ab35c640 1 -- start start 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14ab35c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14a40b9110 0x7f14a40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14ab35c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4072f50 0x7f14a40b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14ab35c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a40b1ea0 con 0x7f14a40b94f0 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14ab35c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f14a40b7a30 con 0x7f14a40b9110 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14ab35c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14a40b7c80 con 0x7f14a4072f50 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14a40b9110 0x7f14a40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f14a40b9110 0x7f14a40b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43502/0 (socket says 172.21.15.40:43502) 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 -- 172.21.15.40:0/3079888391 learned_addr learned my addr 172.21.15.40:0/3079888391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a88f7640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4072f50 0x7f14a40b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:10.649 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 -- 172.21.15.40:0/3079888391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4072f50 msgr2=0x7f14a40b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:10.650 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4072f50 0x7f14a40b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.650 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 -- 172.21.15.40:0/3079888391 >> v1:172.21.15.204:6789/0 conn(0x7f14a40b94f0 legacy=0x7f14a4148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:10.650 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 -- 172.21.15.40:0/3079888391 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40b7f60 con 0x7f14a40b9110 2024-04-02T16:11:10.651 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f14a90f8640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b37c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b7bb0 tx=0x7f14a001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d81356c25b2226 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:10.651 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f149bfff640 1 -- 172.21.15.40:0/3079888391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00b0070 con 0x7f14a40b9110 2024-04-02T16:11:10.651 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f149bfff640 1 -- 172.21.15.40:0/3079888391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a00ab600 con 0x7f14a40b9110 2024-04-02T16:11:10.651 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.645+0000 7f149bfff640 1 -- 172.21.15.40:0/3079888391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00ab9d0 con 0x7f14a40b9110 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3079888391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 msgr2=0x7f14a40b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b37c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f14a40b7bb0 tx=0x7f14a001c420 comp rx=0 tx=0).stop 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3079888391 shutdown_connections 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a4072f50 0x7f14a40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3079888391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b37c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3079888391 >> 172.21.15.40:0/3079888391 conn(0x7f14a406c9c0 msgr2=0x7f14a406cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3079888391 shutdown_connections 2024-04-02T16:11:10.652 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3079888391 wait complete. 2024-04-02T16:11:10.653 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 Processor -- start 2024-04-02T16:11:10.653 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- start start 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a4072f50 0x7f14a40711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40b94f0 0x7f14a41e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f14a41527c0 con 0x7f14a40b9110 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14a4152640 con 0x7f14a4072f50 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14a4152940 con 0x7f14a40b94f0 2024-04-02T16:11:10.655 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43512/0 (socket says 172.21.15.40:43512) 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 -- 172.21.15.40:0/3114375827 learned_addr learned my addr 172.21.15.40:0/3114375827 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a90f8640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a4072f50 0x7f14a40711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a98f9640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40b94f0 0x7f14a41e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 -- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40b94f0 msgr2=0x7f14a41e41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40b94f0 0x7f14a41e41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 -- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a4072f50 msgr2=0x7f14a40711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a4072f50 0x7f14a40711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 -- 172.21.15.40:0/3114375827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a40b61b0 con 0x7f14a40b9110 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14a88f7640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f149400d5c0 tx=0x7f149400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940125f0 con 0x7f14a40b9110 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1494019070 con 0x7f14a40b9110 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a4072420 con 0x7f14a40b9110 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14940129d0 con 0x7f14a40b9110 2024-04-02T16:11:10.656 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a40b50f0 con 0x7f14a40b9110 2024-04-02T16:11:10.657 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1494012b70 con 0x7f14a40b9110 2024-04-02T16:11:10.657 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.649+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1494003bc0 con 0x7f14a40b9110 2024-04-02T16:11:10.657 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.653+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a40b64f0 con 0x7f14a40b9110 2024-04-02T16:11:10.657 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.653+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f14a40b64f0 con 0x7f14a40b9110 2024-04-02T16:11:10.717 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14a4068f20 con 0x7f14a40b9110 2024-04-02T16:11:10.718 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f1499ffb640 1 -- 172.21.15.40:0/3114375827 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1494023020 con 0x7f14a40b9110 2024-04-02T16:11:10.718 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-02T16:11:10.718 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:09.416797+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 msgr2=0x7f14a40b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f149400d5c0 tx=0x7f149400da80 comp rx=0 tx=0).stop 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 shutdown_connections 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14a40b94f0 0x7f14a41e41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a40b9110 0x7f14a40b6d20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 --2- 172.21.15.40:0/3114375827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a4072f50 0x7f14a40711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:10.719 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 >> 172.21.15.40:0/3114375827 conn(0x7f14a406c9c0 msgr2=0x7f14a406bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:10.720 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 shutdown_connections 2024-04-02T16:11:10.720 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:10.713+0000 7f14ab35c640 1 -- 172.21.15.40:0/3114375827 wait complete. 2024-04-02T16:11:10.734 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-02T16:11:11.134 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.129+0000 7f75cedb8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T16:11:11.209 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.205+0000 7f75cedb8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T16:11:11.241 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.237+0000 7f3a5c26d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T16:11:11.319 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.313+0000 7f3a5c26d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T16:11:11.390 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.385+0000 7f75cedb8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T16:11:11.463 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.457+0000 7f75cedb8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T16:11:11.509 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.505+0000 7f3a5c26d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T16:11:11.586 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.581+0000 7f3a5c26d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T16:11:11.751 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.745+0000 7f75cedb8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T16:11:11.811 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:11.805+0000 7f75cedb8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T16:11:11.897 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.893+0000 7f3a5c26d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T16:11:11.962 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-04-02T16:11:11.957+0000 7f3a5c26d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T16:11:12.850 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:12.845+0000 7f7576201640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-02T16:11:12.851 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-04-02T16:11:12.845+0000 7f7576201640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-02T16:11:13.853 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-02T16:11:13.849+0000 7f12f52617c0 -1 osd.3 0 log_to_monitors true 2024-04-02T16:11:13.949 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-02T16:11:13.945+0000 7f11fd54c7c0 -1 osd.5 0 log_to_monitors true 2024-04-02T16:11:13.954 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-02T16:11:13.949+0000 7f01ec8327c0 -1 osd.4 0 log_to_monitors true 2024-04-02T16:11:13.979 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-02T16:11:13.973+0000 7fc54d1147c0 -1 osd.1 0 log_to_monitors true 2024-04-02T16:11:13.985 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-02T16:11:13.981+0000 7f892745c7c0 -1 osd.7 0 log_to_monitors true 2024-04-02T16:11:13.998 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-02T16:11:13.993+0000 7f37284827c0 -1 osd.6 0 log_to_monitors true 2024-04-02T16:11:14.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-02T16:11:14.021+0000 7f454b53b7c0 -1 osd.0 0 log_to_monitors true 2024-04-02T16:11:14.056 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-02T16:11:14.053+0000 7f27196df7c0 -1 osd.2 0 log_to_monitors true 2024-04-02T16:11:17.542 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:17.720 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 Processor -- start 2024-04-02T16:11:17.722 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 -- start start 2024-04-02T16:11:17.722 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 0x7f19b414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.722 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19b40b5da0 0x7f19b414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b40ba7c0 con 0x7f19b4141340 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f19b40b4c30 con 0x7f19b40b5da0 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19bc35c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19b40b4600 con 0x7f19b40ba3e0 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba8f9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f19b4141340 0x7f19b41417d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53588/0 (socket says 172.21.15.40:53588) 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba8f9640 1 -- 172.21.15.40:0/3277547809 learned_addr learned my addr 172.21.15.40:0/3277547809 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/3277547809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 0x7f19b414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19b90f6640 1 -- 172.21.15.40:0/3277547809 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143768377 0 0) 0x7f19b40ba7c0 con 0x7f19b4141340 2024-04-02T16:11:17.723 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19b90f6640 1 -- 172.21.15.40:0/3277547809 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19a0003530 con 0x7f19b4141340 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba0f8640 1 -- 172.21.15.40:0/3277547809 >> v2:172.21.15.204:3300/0 conn(0x7f19b40b5da0 msgr2=0x7f19b414b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/3277547809 >> v2:172.21.15.204:3300/0 conn(0x7f19b40b5da0 0x7f19b414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba0f8640 1 -- 172.21.15.40:0/3277547809 >> v1:172.21.15.40:6789/0 conn(0x7f19b4141340 legacy=0x7f19b41417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.717+0000 7f19ba0f8640 1 -- 172.21.15.40:0/3277547809 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b40b48c0 con 0x7f19b40ba3e0 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/3277547809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 0x7f19b414ad60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19b00097c0 tx=0x7f19b001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60580759653655cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19b90f6640 1 -- 172.21.15.40:0/3277547809 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00b1070 con 0x7f19b40ba3e0 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19b90f6640 1 -- 172.21.15.40:0/3277547809 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b00ac600 con 0x7f19b40ba3e0 2024-04-02T16:11:17.726 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19b90f6640 1 -- 172.21.15.40:0/3277547809 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b00ac9b0 con 0x7f19b40ba3e0 2024-04-02T16:11:17.727 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 -- 172.21.15.40:0/3277547809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 msgr2=0x7f19b414ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:17.727 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 --2- 172.21.15.40:0/3277547809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 0x7f19b414ad60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19b00097c0 tx=0x7f19b001c380 comp rx=0 tx=0).stop 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 -- 172.21.15.40:0/3277547809 shutdown_connections 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 --2- 172.21.15.40:0/3277547809 >> v2:172.21.15.204:3300/0 conn(0x7f19b40b5da0 0x7f19b414b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 --2- 172.21.15.40:0/3277547809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b40ba3e0 0x7f19b414ad60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.721+0000 7f19bc35c640 1 -- 172.21.15.40:0/3277547809 >> 172.21.15.40:0/3277547809 conn(0x7f19b40bd5f0 msgr2=0x7f19b40b9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- 172.21.15.40:0/3277547809 shutdown_connections 2024-04-02T16:11:17.728 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- 172.21.15.40:0/3277547809 wait complete. 2024-04-02T16:11:17.729 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 Processor -- start 2024-04-02T16:11:17.729 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- start start 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b4141340 0x7f19b4143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 0x7f19b4143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4154830 con 0x7f19b41e3b30 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19b40b9810 con 0x7f19b40ba3e0 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19bc35c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19b41549b0 con 0x7f19b4141340 2024-04-02T16:11:17.731 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19ba0f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:17.732 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19ba0f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:52292/0 (socket says 172.21.15.40:52292) 2024-04-02T16:11:17.732 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19ba0f8640 1 -- 172.21.15.40:0/2066425294 learned_addr learned my addr 172.21.15.40:0/2066425294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:17.732 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19ba8f9640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 0x7f19b4143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:17.732 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.725+0000 7f19b98f7640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b4141340 0x7f19b4143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 -- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b4141340 msgr2=0x7f19b4143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b4141340 0x7f19b4143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 -- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 msgr2=0x7f19b41e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 -- 172.21.15.40:0/2066425294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b4144850 con 0x7f19b41e3b30 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba8f9640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 0x7f19b4143750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00caa0 tx=0x7f19ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac017070 con 0x7f19b41e3b30 2024-04-02T16:11:17.733 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ac00ab40 con 0x7f19b41e3b30 2024-04-02T16:11:17.734 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac012400 con 0x7f19b41e3b30 2024-04-02T16:11:17.734 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b4145060 con 0x7f19b41e3b30 2024-04-02T16:11:17.734 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b4144ac0 con 0x7f19b41e3b30 2024-04-02T16:11:17.734 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b4068f20 con 0x7f19b41e3b30 2024-04-02T16:11:17.734 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) ==== 87931+0+0 (secure 0 0 0) 0x7f19ac01a070 con 0x7f19b41e3b30 2024-04-02T16:11:17.735 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19aaffd640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 0x7f199806a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:17.735 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.729+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 0x7f199806a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:17.736 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.733+0000 7f19ba0f8640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 0x7f199806a7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19b001aa30 tx=0x7f19b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:17.740 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.737+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f19ac054300 con 0x7f19b41e3b30 2024-04-02T16:11:17.741 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:17.737+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f19ac056260 con 0x7f19b41e3b30 2024-04-02T16:11:17.964 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-02T16:11:17.961+0000 7f2715688640 -1 osd.2 0 waiting for initial osdmap 2024-04-02T16:11:17.999 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-04-02T16:11:17.993+0000 7f270d4bc640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.006 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-02T16:11:18.001+0000 7f372442b640 -1 osd.6 0 waiting for initial osdmap 2024-04-02T16:11:18.012 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-02T16:11:18.009+0000 7f11f94f5640 -1 osd.5 0 waiting for initial osdmap 2024-04-02T16:11:18.025 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-02T16:11:18.021+0000 7f8923405640 -1 osd.7 0 waiting for initial osdmap 2024-04-02T16:11:18.040 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-04-02T16:11:18.037+0000 7f371c25f640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.043 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-02T16:11:18.041+0000 7f01e87db640 -1 osd.4 0 waiting for initial osdmap 2024-04-02T16:11:18.047 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-02T16:11:18.041+0000 7f12f120a640 -1 osd.3 0 waiting for initial osdmap 2024-04-02T16:11:18.052 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-04-02T16:11:18.049+0000 7f11f1329640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.056 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19b4140dd0 con 0x7f19b41e3b30 2024-04-02T16:11:18.057 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19aaffd640 1 -- 172.21.15.40:0/2066425294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f19ac05a560 con 0x7f19b41e3b30 2024-04-02T16:11:18.057 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-02T16:11:18.058 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":12,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:15.853438+0000","last_up_change":"0.000000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 msgr2=0x7f199806a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 0x7f199806a7b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19b001aa30 tx=0x7f19b0008040 comp rx=0 tx=0).stop 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 msgr2=0x7f19b4143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 0x7f19b4143750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00caa0 tx=0x7f19ac00cf60 comp rx=0 tx=0).stop 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 shutdown_connections 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19b4141340 0x7f19b4143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19980682f0 0x7f199806a7b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:18.060 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19b41e3b30 0x7f19b4143750 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:18.061 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 --2- 172.21.15.40:0/2066425294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19b40ba3e0 0x7f19b41e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:18.061 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 >> 172.21.15.40:0/2066425294 conn(0x7f19b40bd5f0 msgr2=0x7f19b40b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:18.061 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 shutdown_connections 2024-04-02T16:11:18.061 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:18.053+0000 7f19bc35c640 1 -- 172.21.15.40:0/2066425294 wait complete. 2024-04-02T16:11:18.068 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-04-02T16:11:18.065+0000 7f01e060f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.075 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-02T16:11:18.076 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-04-02T16:11:18.073+0000 7f891b239640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.088 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-04-02T16:11:18.085+0000 7f12e903e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.131 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-02T16:11:18.125+0000 7fc5490bd640 -1 osd.1 0 waiting for initial osdmap 2024-04-02T16:11:18.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-02T16:11:18.149+0000 7f4547cf6640 -1 osd.0 0 waiting for initial osdmap 2024-04-02T16:11:18.156 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-04-02T16:11:18.153+0000 7fc540ef1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:18.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-04-02T16:11:18.197+0000 7f453f318640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T16:11:24.882 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:25.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 Processor -- start 2024-04-02T16:11:25.008 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 -- start start 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 0x7ff9500af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff950147c20 0x7ff95006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff950148070 con 0x7ff95006cb20 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff9500b4eb0 con 0x7ff950147c20 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff955ced640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff9500b0ba0 con 0x7ff9500b4ad0 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff950147c20 0x7ff95006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.010 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff950147c20 0x7ff95006c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36310/0 (socket says 172.21.15.40:36310) 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 -- 172.21.15.40:0/2078175567 learned_addr learned my addr 172.21.15.40:0/2078175567 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94f7fe640 1 --2- 172.21.15.40:0/2078175567 >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 0x7ff9500af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 -- 172.21.15.40:0/2078175567 >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 msgr2=0x7ff9500af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 --2- 172.21.15.40:0/2078175567 >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 0x7ff9500af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 -- 172.21.15.40:0/2078175567 >> v1:172.21.15.204:6789/0 conn(0x7ff95006cb20 legacy=0x7ff950146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94effd640 1 -- 172.21.15.40:0/2078175567 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500add80 con 0x7ff950147c20 2024-04-02T16:11:25.011 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.005+0000 7ff94f7fe640 1 --2- 172.21.15.40:0/2078175567 >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 0x7ff9500af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:25.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff94effd640 1 --2- 172.21.15.40:0/2078175567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95006c5e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff95006e2c0 tx=0x7ff94001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428ad9324948a94e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.014 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff94e7fc640 1 -- 172.21.15.40:0/2078175567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400aa610 con 0x7ff950147c20 2024-04-02T16:11:25.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff94e7fc640 1 -- 172.21.15.40:0/2078175567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9400af070 con 0x7ff950147c20 2024-04-02T16:11:25.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff94e7fc640 1 -- 172.21.15.40:0/2078175567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9400aa9d0 con 0x7ff950147c20 2024-04-02T16:11:25.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 -- 172.21.15.40:0/2078175567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 msgr2=0x7ff95006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.015 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 --2- 172.21.15.40:0/2078175567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95006c5e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff95006e2c0 tx=0x7ff94001c490 comp rx=0 tx=0).stop 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 -- 172.21.15.40:0/2078175567 shutdown_connections 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 --2- 172.21.15.40:0/2078175567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95006c5e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 --2- 172.21.15.40:0/2078175567 >> v2:172.21.15.204:3300/0 conn(0x7ff9500b4ad0 0x7ff9500af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 -- 172.21.15.40:0/2078175567 >> 172.21.15.40:0/2078175567 conn(0x7ff9500b8f30 msgr2=0x7ff9500b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 -- 172.21.15.40:0/2078175567 shutdown_connections 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.009+0000 7ff955ced640 1 -- 172.21.15.40:0/2078175567 wait complete. 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 Processor -- start 2024-04-02T16:11:25.016 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- start start 2024-04-02T16:11:25.018 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff95006cb20 0x7ff950141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.018 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9500b4ad0 0x7ff950142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.018 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.018 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006db90 con 0x7ff950147c20 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff95006d340 con 0x7ff9500b4ad0 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff95006dd10 con 0x7ff95006cb20 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36322/0 (socket says 172.21.15.40:36322) 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 -- 172.21.15.40:0/649412698 learned_addr learned my addr 172.21.15.40:0/649412698 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94f7fe640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff95006cb20 0x7ff950141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94effd640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9500b4ad0 0x7ff950142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 -- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff95006cb20 msgr2=0x7ff950141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff95006cb20 0x7ff950141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 -- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9500b4ad0 msgr2=0x7ff950142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9500b4ad0 0x7ff950142750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 -- 172.21.15.40:0/649412698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff950143060 con 0x7ff950147c20 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff94ffff640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c730 tx=0x7ff94800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.019 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948003b80 con 0x7ff950147c20 2024-04-02T16:11:25.020 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff948003d20 con 0x7ff950147c20 2024-04-02T16:11:25.020 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94800aa30 con 0x7ff950147c20 2024-04-02T16:11:25.020 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff950143340 con 0x7ff950147c20 2024-04-02T16:11:25.020 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff950142c90 con 0x7ff950147c20 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.013+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950068f20 con 0x7ff950147c20 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.017+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff94800abd0 con 0x7ff950147c20 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.017+0000 7ff947fff640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 0x7ff93006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.017+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff948086460 con 0x7ff950147c20 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.017+0000 7ff94f7fe640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 0x7ff93006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.021 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.017+0000 7ff94f7fe640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 0x7ff93006a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9380096d0 tx=0x7ff938008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.025 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.021+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff948056e00 con 0x7ff950147c20 2024-04-02T16:11:25.191 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.185+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9500b5270 con 0x7ff950147c20 2024-04-02T16:11:25.193 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff947fff640 1 -- 172.21.15.40:0/649412698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7ff948056700 con 0x7ff950147c20 2024-04-02T16:11:25.193 INFO:teuthology.misc.health.smithi040.stdout: 2024-04-02T16:11:25.193 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":16,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:22.047301+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:25.195 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 msgr2=0x7ff93006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.195 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 0x7ff93006a840 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9380096d0 tx=0x7ff938008040 comp rx=0 tx=0).stop 2024-04-02T16:11:25.195 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 msgr2=0x7ff95009c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff94800c730 tx=0x7ff94800cbf0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 shutdown_connections 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff930068380 0x7ff93006a840 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff950147c20 0x7ff95009c900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9500b4ad0 0x7ff950142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 --2- 172.21.15.40:0/649412698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff95006cb20 0x7ff950141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 >> 172.21.15.40:0/649412698 conn(0x7ff9500b8f30 msgr2=0x7ff9500b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 shutdown_connections 2024-04-02T16:11:25.196 INFO:teuthology.misc.health.smithi040.stderr:2024-04-02T16:11:25.189+0000 7ff955ced640 1 -- 172.21.15.40:0/649412698 wait complete. 2024-04-02T16:11:25.212 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-02T16:11:25.212 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-02T16:11:25.212 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-02T16:11:25.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-02T16:11:25.215 INFO:tasks.ceph.mds.a:Started 2024-04-02T16:11:25.215 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-02T16:11:25.215 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-02T16:11:25.218 INFO:tasks.ceph.mds.b:Started 2024-04-02T16:11:25.218 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-02T16:11:25.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:25.291 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-02T16:11:25.307 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-02T16:11:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 Processor -- start 2024-04-02T16:11:25.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 -- start start 2024-04-02T16:11:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b4bf0 0x7fe4dc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4dc0b6310 con 0x7fe4dc0b3500 2024-04-02T16:11:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe4dc0b55d0 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4e1c91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc14b560 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36062/0 (socket says 172.21.15.40:36062) 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 -- 172.21.15.40:0/1043098219 learned_addr learned my addr 172.21.15.40:0/1043098219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4db7fe640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b4bf0 0x7fe4dc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 -- 172.21.15.40:0/1043098219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b4bf0 msgr2=0x7fe4dc0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b4bf0 0x7fe4dc0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 -- 172.21.15.40:0/1043098219 >> v1:172.21.15.204:6789/0 conn(0x7fe4dc0b3500 legacy=0x7fe4dc14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:25.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.337+0000 7fe4dbfff640 1 -- 172.21.15.40:0/1043098219 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc0b3fa0 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4dbfff640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009e50 tx=0x7fe4d401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31f66797887e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4da7fc640 1 -- 172.21.15.40:0/1043098219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a5550 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4da7fc640 1 -- 172.21.15.40:0/1043098219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40a56f0 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4da7fc640 1 -- 172.21.15.40:0/1043098219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a5ac0 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- 172.21.15.40:0/1043098219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b8c80 msgr2=0x7fe4dc0b9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009e50 tx=0x7fe4d401c8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- 172.21.15.40:0/1043098219 shutdown_connections 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b8c80 0x7fe4dc0b9110 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/1043098219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b4bf0 0x7fe4dc0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- 172.21.15.40:0/1043098219 >> 172.21.15.40:0/1043098219 conn(0x7fe4dc0734b0 msgr2=0x7fe4dc0bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- 172.21.15.40:0/1043098219 shutdown_connections 2024-04-02T16:11:25.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- 172.21.15.40:0/1043098219 wait complete. 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 Processor -- start 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- start start 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 0x7fe4dc144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b8c80 0x7fe4dc142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc154d90 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc154c10 con 0x7fe4dc0b3500 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4e1c91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe4dc154f10 con 0x7fe4dc0b8c80 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36066/0 (socket says 172.21.15.40:36066) 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 -- 172.21.15.40:0/3300471459 learned_addr learned my addr 172.21.15.40:0/3300471459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4dbfff640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b8c80 0x7fe4dc142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4db7fe640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 0x7fe4dc144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 -- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b8c80 msgr2=0x7fe4dc142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b8c80 0x7fe4dc142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 -- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 msgr2=0x7fe4dc144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 0x7fe4dc144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 -- 172.21.15.40:0/3300471459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc14bd10 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4db7fe640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 0x7fe4dc144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4daffd640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000c480 tx=0x7fe4d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0012830 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.341+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d0017070 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc148240 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0012c30 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc143ff0 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc0685f0 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe4d001c460 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4c3fff640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 0x7fe4c806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe4d0086340 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4db7fe640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 0x7fe4c806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.345+0000 7fe4db7fe640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 0x7fe4c806a870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc007a70 tx=0x7fe4cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.349+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fe4d0056ce0 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.513+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4dc13db90 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.513+0000 7fe4c3fff640 1 -- 172.21.15.40:0/3300471459 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7fe4d00565e0 con 0x7fe4dc0b4bf0 2024-04-02T16:11:25.519 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:25.519 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:22.047301+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 msgr2=0x7fe4c806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 0x7fe4c806a870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc007a70 tx=0x7fe4cc008040 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 msgr2=0x7fe4dc151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000c480 tx=0x7fe4d000c940 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 shutdown_connections 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4dc0b8c80 0x7fe4dc142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe4c80683b0 0x7fe4c806a870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4dc0b4bf0 0x7fe4dc151840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 --2- 172.21.15.40:0/3300471459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4dc0b3500 0x7fe4dc144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 >> 172.21.15.40:0/3300471459 conn(0x7fe4dc0734b0 msgr2=0x7fe4dc141770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 shutdown_connections 2024-04-02T16:11:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.517+0000 7fe4e1c91640 1 -- 172.21.15.40:0/3300471459 wait complete. 2024-04-02T16:11:25.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:25.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:25.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.665+0000 7f922751d640 1 Processor -- start 2024-04-02T16:11:25.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.665+0000 7f922751d640 1 -- start start 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f922751d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9220072910 0x7f92200af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f922751d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9220148550 0x7f9220070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f922751d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9220147a70 con 0x7f9220071380 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f922751d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9220147d50 con 0x7f9220072910 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f922751d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9220148030 con 0x7f9220148550 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9225aba640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9220071380 0x7f9220141460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49130/0 (socket says 172.21.15.40:49130) 2024-04-02T16:11:25.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9220072910 0x7f92200af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9225aba640 1 -- 172.21.15.40:0/3362273942 learned_addr learned my addr 172.21.15.40:0/3362273942 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9217fff640 1 -- 172.21.15.40:0/3362273942 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351697395 0 0) 0x7f9220147a70 con 0x7f9220071380 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9217fff640 1 -- 172.21.15.40:0/3362273942 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f920c003530 con 0x7f9220071380 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9224ab8640 1 --2- 172.21.15.40:0/3362273942 >> v2:172.21.15.204:3300/0 conn(0x7f9220148550 0x7f9220070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 -- 172.21.15.40:0/3362273942 >> v2:172.21.15.204:3300/0 conn(0x7f9220148550 msgr2=0x7f9220070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 --2- 172.21.15.40:0/3362273942 >> v2:172.21.15.204:3300/0 conn(0x7f9220148550 0x7f9220070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 -- 172.21.15.40:0/3362273942 >> v1:172.21.15.40:6789/0 conn(0x7f9220071380 legacy=0x7f9220141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 -- 172.21.15.40:0/3362273942 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9220072cf0 con 0x7f9220072910 2024-04-02T16:11:25.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f92252b9640 1 --2- 172.21.15.40:0/3362273942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200af1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92200b44d0 tx=0x7f921c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b93f5006f38d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9217fff640 1 -- 172.21.15.40:0/3362273942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0b0070 con 0x7f9220072910 2024-04-02T16:11:25.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9217fff640 1 -- 172.21.15.40:0/3362273942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c0ab600 con 0x7f9220072910 2024-04-02T16:11:25.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.669+0000 7f9217fff640 1 -- 172.21.15.40:0/3362273942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0ab9d0 con 0x7f9220072910 2024-04-02T16:11:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/3362273942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 msgr2=0x7f92200af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- 172.21.15.40:0/3362273942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200af1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92200b44d0 tx=0x7f921c01c360 comp rx=0 tx=0).stop 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/3362273942 shutdown_connections 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- 172.21.15.40:0/3362273942 >> v2:172.21.15.204:3300/0 conn(0x7f9220148550 0x7f9220070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- 172.21.15.40:0/3362273942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200af1e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/3362273942 >> 172.21.15.40:0/3362273942 conn(0x7f92200b7f50 msgr2=0x7f92200b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/3362273942 shutdown_connections 2024-04-02T16:11:25.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/3362273942 wait complete. 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 Processor -- start 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- start start 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 0x7f92201dde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9220136fe0 0x7f9220134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f922014f8b0 con 0x7f9220072910 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f922014f730 con 0x7f9220136fe0 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f922014fa30 con 0x7f9220148550 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36078/0 (socket says 172.21.15.40:36078) 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 -- 172.21.15.40:0/4168729445 learned_addr learned my addr 172.21.15.40:0/4168729445 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9224ab8640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 0x7f92201dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9225aba640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9220136fe0 0x7f9220134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 -- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 msgr2=0x7f92201dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 0x7f92201dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 -- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9220136fe0 msgr2=0x7f9220134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9220136fe0 0x7f9220134ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 -- 172.21.15.40:0/4168729445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92200b6570 con 0x7f9220072910 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9224ab8640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 0x7f92201dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f92252b9640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f921c0ae040 tx=0x7f921c01aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0b0050 con 0x7f9220072910 2024-04-02T16:11:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c0bc070 con 0x7f9220072910 2024-04-02T16:11:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9220139d40 con 0x7f9220072910 2024-04-02T16:11:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0b5a30 con 0x7f9220072910 2024-04-02T16:11:25.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.673+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201de3b0 con 0x7f9220072910 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9220068d00 con 0x7f9220072910 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f921c0b5bd0 con 0x7f9220072910 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f9215ffb640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 0x7f920406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f921c120f80 con 0x7f9220072910 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f9224ab8640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 0x7f920406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.677+0000 7f9224ab8640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 0x7f920406a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9210009840 tx=0x7f9210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.681+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f921c016070 con 0x7f9220072910 2024-04-02T16:11:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.841+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f922013fbd0 con 0x7f9220072910 2024-04-02T16:11:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.841+0000 7f9215ffb640 1 -- 172.21.15.40:0/4168729445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f921c0f1920 con 0x7f9220072910 2024-04-02T16:11:25.848 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 msgr2=0x7f920406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 0x7f920406a840 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9210009840 tx=0x7f9210008040 comp rx=0 tx=0).stop 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 msgr2=0x7f92200b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f921c0ae040 tx=0x7f921c01aba0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 shutdown_connections 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9220148550 0x7f92201dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9204068380 0x7f920406a840 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9220136fe0 0x7f9220134ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 --2- 172.21.15.40:0/4168729445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9220072910 0x7f92200b5880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 >> 172.21.15.40:0/4168729445 conn(0x7f92200b7f50 msgr2=0x7f9220135c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 shutdown_connections 2024-04-02T16:11:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.845+0000 7f922751d640 1 -- 172.21.15.40:0/4168729445 wait complete. 2024-04-02T16:11:25.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.981+0000 7f31c4907640 1 Processor -- start 2024-04-02T16:11:25.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.981+0000 7f31c4907640 1 -- start start 2024-04-02T16:11:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c0072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31c0145550 0x7f31c0145990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c00b2e40 con 0x7f31c00718d0 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f31c00b3e30 con 0x7f31c0145550 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31c00b30b0 con 0x7f31c00b1d40 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f31c0145550 0x7f31c0145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f31c00718d0 0x7f31c0071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49138/0 (socket says 172.21.15.40:49138) 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bed9d640 1 -- 172.21.15.40:0/278536124 learned_addr learned my addr 172.21.15.40:0/278536124 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31be59c640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c0072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 -- 172.21.15.40:0/278536124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 msgr2=0x7f31c0072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c0072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 -- 172.21.15.40:0/278536124 >> v1:172.21.15.40:6789/0 conn(0x7f31c00718d0 legacy=0x7f31c0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 -- 172.21.15.40:0/278536124 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c00b42c0 con 0x7f31c0145550 2024-04-02T16:11:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bd59a640 1 -- 172.21.15.40:0/278536124 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980059055 0 0) 0x7f31c00b2e40 con 0x7f31c00718d0 2024-04-02T16:11:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31be59c640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c0072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bdd9b640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c0145550 0x7f31c0145990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31b40097c0 tx=0x7f31b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c3f5fda7892e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bd59a640 1 -- 172.21.15.40:0/278536124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40b0070 con 0x7f31c0145550 2024-04-02T16:11:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bd59a640 1 -- 172.21.15.40:0/278536124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b40ab600 con 0x7f31c0145550 2024-04-02T16:11:25.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31bd59a640 1 -- 172.21.15.40:0/278536124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40ab9d0 con 0x7f31c0145550 2024-04-02T16:11:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 -- 172.21.15.40:0/278536124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c0145550 msgr2=0x7f31c0145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.985+0000 7f31c4907640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c0145550 0x7f31c0145990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31b40097c0 tx=0x7f31b401c4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/278536124 shutdown_connections 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c0145550 0x7f31c0145990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 --2- 172.21.15.40:0/278536124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c0072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/278536124 >> 172.21.15.40:0/278536124 conn(0x7f31c00b7d10 msgr2=0x7f31c00b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/278536124 shutdown_connections 2024-04-02T16:11:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/278536124 wait complete. 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 Processor -- start 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- start start 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31c0145550 0x7f31c0140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0152610 con 0x7f31c00718d0 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f31c00b3820 con 0x7f31c0145550 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31c0152790 con 0x7f31c00b1d40 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36090/0 (socket says 172.21.15.40:36090) 2024-04-02T16:11:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 -- 172.21.15.40:0/2591508291 learned_addr learned my addr 172.21.15.40:0/2591508291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31bdd9b640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31bed9d640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31c0145550 0x7f31c0140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 -- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 msgr2=0x7f31c013fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c013fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 -- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31c0145550 msgr2=0x7f31c0140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31c0145550 0x7f31c0140400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 -- 172.21.15.40:0/2591508291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c013d720 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31be59c640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e840 tx=0x7f31b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0017070 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b000c8d0 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c013da00 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b000f040 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c006e030 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.989+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c0068d00 con 0x7f31c00718d0 2024-04-02T16:11:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.993+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f31b001a070 con 0x7f31c00718d0 2024-04-02T16:11:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.993+0000 7f31aeffd640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 0x7f31a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.993+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f31b00862a0 con 0x7f31c00718d0 2024-04-02T16:11:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.993+0000 7f31bdd9b640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 0x7f31a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.993+0000 7f31bdd9b640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 0x7f31a006a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ad000 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:26.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:25.997+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f31b0056ba0 con 0x7f31c00718d0 2024-04-02T16:11:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.197+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31c0142bd0 con 0x7f31c00718d0 2024-04-02T16:11:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.197+0000 7f31aeffd640 1 -- 172.21.15.40:0/2591508291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f31b00564a0 con 0x7f31c00718d0 2024-04-02T16:11:26.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:26.202 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T16:11:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-04-02T16:11:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 msgr2=0x7f31a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 0x7f31a006a840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f31b40ad000 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2024-04-02T16:11:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 msgr2=0x7f31c006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f31b000e840 tx=0x7f31b000ed00 comp rx=0 tx=0).stop 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 shutdown_connections 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31c00b1d40 0x7f31c013fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f31a0068380 0x7f31a006a840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31c0145550 0x7f31c0140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 --2- 172.21.15.40:0/2591508291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31c00718d0 0x7f31c006d380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 >> 172.21.15.40:0/2591508291 conn(0x7f31c00b7d10 msgr2=0x7f31c0137df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 shutdown_connections 2024-04-02T16:11:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:26.201+0000 7f31c4907640 1 -- 172.21.15.40:0/2591508291 wait complete. 2024-04-02T16:11:28.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:28.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 Processor -- start 2024-04-02T16:11:28.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 -- start start 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3af006c8a0 0x7f3af00af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3af00b43a0 0x7f3af014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af00ae830 con 0x7f3af00b3800 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3af00b2b50 con 0x7f3af00b43a0 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3af5b8d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3af00b0300 con 0x7f3af006c8a0 2024-04-02T16:11:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3af00b43a0 0x7f3af014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3af00b3800 0x7f3af00b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49150/0 (socket says 172.21.15.40:49150) 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeeffd640 1 -- 172.21.15.40:0/501245033 learned_addr learned my addr 172.21.15.40:0/501245033 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aef7fe640 1 --2- 172.21.15.40:0/501245033 >> v2:172.21.15.204:3300/0 conn(0x7f3af006c8a0 0x7f3af00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeffff640 1 -- 172.21.15.40:0/501245033 >> v2:172.21.15.204:3300/0 conn(0x7f3af006c8a0 msgr2=0x7f3af00af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeffff640 1 --2- 172.21.15.40:0/501245033 >> v2:172.21.15.204:3300/0 conn(0x7f3af006c8a0 0x7f3af00af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeffff640 1 -- 172.21.15.40:0/501245033 >> v1:172.21.15.40:6789/0 conn(0x7f3af00b3800 legacy=0x7f3af00b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aeffff640 1 -- 172.21.15.40:0/501245033 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af006cc80 con 0x7f3af00b43a0 2024-04-02T16:11:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.349+0000 7f3aee7fc640 1 -- 172.21.15.40:0/501245033 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249548340 0 0) 0x7f3af00ae830 con 0x7f3af00b3800 2024-04-02T16:11:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aeffff640 1 --2- 172.21.15.40:0/501245033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af00b43a0 0x7f3af014f0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3af006e070 tx=0x7f3ae801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d73bcd83d18c68a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aee7fc640 1 -- 172.21.15.40:0/501245033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80b2070 con 0x7f3af00b43a0 2024-04-02T16:11:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aee7fc640 1 -- 172.21.15.40:0/501245033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae80a5850 con 0x7f3af00b43a0 2024-04-02T16:11:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aee7fc640 1 -- 172.21.15.40:0/501245033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80a5c20 con 0x7f3af00b43a0 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/501245033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af00b43a0 msgr2=0x7f3af014f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/501245033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af00b43a0 0x7f3af014f0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3af006e070 tx=0x7f3ae801c8a0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/501245033 shutdown_connections 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/501245033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af00b43a0 0x7f3af014f0b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/501245033 >> v2:172.21.15.204:3300/0 conn(0x7f3af006c8a0 0x7f3af00af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/501245033 >> 172.21.15.40:0/501245033 conn(0x7f3af00b8fd0 msgr2=0x7f3af00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/501245033 shutdown_connections 2024-04-02T16:11:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/501245033 wait complete. 2024-04-02T16:11:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 Processor -- start 2024-04-02T16:11:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- start start 2024-04-02T16:11:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3af00b3800 0x7f3af014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3af00b43a0 0x7f3af01e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00b3130 con 0x7f3af006c8a0 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00af690 con 0x7f3af00b43a0 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3af0070bf0 con 0x7f3af00b3800 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36110/0 (socket says 172.21.15.40:36110) 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 -- 172.21.15.40:0/2415904355 learned_addr learned my addr 172.21.15.40:0/2415904355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aeffff640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3af00b43a0 0x7f3af01e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aeeffd640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3af00b3800 0x7f3af014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 -- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3af00b3800 msgr2=0x7f3af014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3af00b3800 0x7f3af014dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 -- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3af00b43a0 msgr2=0x7f3af01e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3af00b43a0 0x7f3af01e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 -- 172.21.15.40:0/2415904355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af014e7c0 con 0x7f3af006c8a0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3aef7fe640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000d830 tx=0x7f3ae000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000b6c0 con 0x7f3af006c8a0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0019070 con 0x7f3af006c8a0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af0070e40 con 0x7f3af006c8a0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000ba60 con 0x7f3af006c8a0 2024-04-02T16:11:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.353+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af0124b40 con 0x7f3af006c8a0 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af00685f0 con 0x7f3af006c8a0 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ae001e460 con 0x7f3af006c8a0 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3ad7fff640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 0x7f3adc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3ae0086ca0 con 0x7f3af006c8a0 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3aeeffd640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 0x7f3adc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.357+0000 7f3aeeffd640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 0x7f3adc06a870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4007fd0 tx=0x7f3ae4007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.361+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3ae0057640 con 0x7f3af006c8a0 2024-04-02T16:11:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.553+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3af01e72d0 con 0x7f3af006c8a0 2024-04-02T16:11:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.553+0000 7f3ad7fff640 1 -- 172.21.15.40:0/2415904355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2213 (secure 0 0 0) 0x7f3ae0056f40 con 0x7f3af006c8a0 2024-04-02T16:11:28.559 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T16:11:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 msgr2=0x7f3adc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 0x7f3adc06a870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4007fd0 tx=0x7f3ae4007ba0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 msgr2=0x7f3af0148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000d830 tx=0x7f3ae000dcf0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 shutdown_connections 2024-04-02T16:11:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3af00b3800 0x7f3af014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3adc0683b0 0x7f3adc06a870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3af00b43a0 0x7f3af01e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 --2- 172.21.15.40:0/2415904355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3af006c8a0 0x7f3af0148400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 >> 172.21.15.40:0/2415904355 conn(0x7f3af00b8fd0 msgr2=0x7f3af00672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 shutdown_connections 2024-04-02T16:11:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.557+0000 7f3af5b8d640 1 -- 172.21.15.40:0/2415904355 wait complete. 2024-04-02T16:11:28.577 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:28.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.685+0000 7f2f3d10c640 1 Processor -- start 2024-04-02T16:11:28.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.685+0000 7f2f3d10c640 1 -- start start 2024-04-02T16:11:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f380b23d0 0x7f2f380b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 0x7f2f38149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f380b19e0 con 0x7f2f38073230 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2f381497c0 con 0x7f2f380b23d0 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f380b05c0 con 0x7f2f38148e50 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f380b23d0 0x7f2f380b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2f38073230 0x7f2f3809daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49154/0 (socket says 172.21.15.40:49154) 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3659c640 1 -- 172.21.15.40:0/2845356474 learned_addr learned my addr 172.21.15.40:0/2845356474 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3759e640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 0x7f2f38149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f35d9b640 1 -- 172.21.15.40:0/2845356474 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635947639 0 0) 0x7f2f380b19e0 con 0x7f2f38073230 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f35d9b640 1 -- 172.21.15.40:0/2845356474 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f20003530 con 0x7f2f38073230 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 -- 172.21.15.40:0/2845356474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 msgr2=0x7f2f38149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 0x7f2f38149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 -- 172.21.15.40:0/2845356474 >> v1:172.21.15.40:6789/0 conn(0x7f2f38073230 legacy=0x7f2f3809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 -- 172.21.15.40:0/2845356474 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f380b08a0 con 0x7f2f380b23d0 2024-04-02T16:11:28.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3759e640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 0x7f2f38149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:28.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f36d9d640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f380b23d0 0x7f2f380b27b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f280097c0 tx=0x7f2f2801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e9861bff5ff31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f35d9b640 1 -- 172.21.15.40:0/2845356474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f280ab430 con 0x7f2f380b23d0 2024-04-02T16:11:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f35d9b640 1 -- 172.21.15.40:0/2845356474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f280b0070 con 0x7f2f380b23d0 2024-04-02T16:11:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f35d9b640 1 -- 172.21.15.40:0/2845356474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f280ab7f0 con 0x7f2f380b23d0 2024-04-02T16:11:28.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2845356474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f380b23d0 msgr2=0x7f2f380b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f380b23d0 0x7f2f380b27b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f280097c0 tx=0x7f2f2801c410 comp rx=0 tx=0).stop 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2845356474 shutdown_connections 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38148e50 0x7f2f38149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2845356474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f380b23d0 0x7f2f380b27b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2845356474 >> 172.21.15.40:0/2845356474 conn(0x7f2f380b8fd0 msgr2=0x7f2f380b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2845356474 shutdown_connections 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.689+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2845356474 wait complete. 2024-04-02T16:11:28.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 Processor -- start 2024-04-02T16:11:28.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- start start 2024-04-02T16:11:28.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38073230 0x7f2f381d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f380b23d0 0x7f2f3813d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38152810 con 0x7f2f38148e50 2024-04-02T16:11:28.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f38152690 con 0x7f2f380b23d0 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f38152990 con 0x7f2f38073230 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36128/0 (socket says 172.21.15.40:36128) 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 -- 172.21.15.40:0/2163066718 learned_addr learned my addr 172.21.15.40:0/2163066718 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 -- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38073230 msgr2=0x7f2f381d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38073230 0x7f2f381d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 -- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f380b23d0 msgr2=0x7f2f3813d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f380b23d0 0x7f2f3813d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 -- 172.21.15.40:0/2163066718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f381de450 con 0x7f2f38148e50 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3759e640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f30007e70 tx=0x7f2f3000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3001a070 con 0x7f2f38148e50 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f30003bc0 con 0x7f2f38148e50 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f381de730 con 0x7f2f38148e50 2024-04-02T16:11:28.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004030 con 0x7f2f38148e50 2024-04-02T16:11:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f380729d0 con 0x7f2f38148e50 2024-04-02T16:11:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f380685f0 con 0x7f2f38148e50 2024-04-02T16:11:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.693+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2f3001d070 con 0x7f2f38148e50 2024-04-02T16:11:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.697+0000 7f2f1f7fe640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 0x7f2f2406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.697+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2f30086890 con 0x7f2f38148e50 2024-04-02T16:11:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.697+0000 7f2f36d9d640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 0x7f2f2406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.697+0000 7f2f36d9d640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 0x7f2f2406a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f280af000 tx=0x7f2f28004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.701+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2f30010070 con 0x7f2f38148e50 2024-04-02T16:11:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.865+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f380b56e0 con 0x7f2f38148e50 2024-04-02T16:11:28.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.865+0000 7f2f1f7fe640 1 -- 172.21.15.40:0/2163066718 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7f2f30057230 con 0x7f2f38148e50 2024-04-02T16:11:28.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:28.870 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:22.047301+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:28.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 msgr2=0x7f2f2406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 0x7f2f2406a840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f280af000 tx=0x7f2f28004740 comp rx=0 tx=0).stop 2024-04-02T16:11:28.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 msgr2=0x7f2f3813df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f30007e70 tx=0x7f2f3000d4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 shutdown_connections 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2f24068380 0x7f2f2406a840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f38148e50 0x7f2f3813df80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f380b23d0 0x7f2f3813d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 --2- 172.21.15.40:0/2163066718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f38073230 0x7f2f381d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 >> 172.21.15.40:0/2163066718 conn(0x7f2f380b8fd0 msgr2=0x7f2f3806cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 shutdown_connections 2024-04-02T16:11:28.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:28.869+0000 7f2f3d10c640 1 -- 172.21.15.40:0/2163066718 wait complete. 2024-04-02T16:11:28.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:28.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 Processor -- start 2024-04-02T16:11:29.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 -- start start 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0874073230 0x7f08740b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0874148e50 0x7f0874149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08740b19e0 con 0x7f08740b23d0 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f08741497c0 con 0x7f0874073230 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f087ae42640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f08740b05c0 con 0x7f0874148e50 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0873fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0874073230 0x7f08740b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0878bde640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f08740b23d0 0x7f087409daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49166/0 (socket says 172.21.15.40:49166) 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0878bde640 1 -- 172.21.15.40:0/2749672055 learned_addr learned my addr 172.21.15.40:0/2749672055 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f08793df640 1 --2- 172.21.15.40:0/2749672055 >> v2:172.21.15.204:3300/0 conn(0x7f0874148e50 0x7f0874149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0873fff640 1 -- 172.21.15.40:0/2749672055 >> v2:172.21.15.204:3300/0 conn(0x7f0874148e50 msgr2=0x7f0874149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0873fff640 1 --2- 172.21.15.40:0/2749672055 >> v2:172.21.15.204:3300/0 conn(0x7f0874148e50 0x7f0874149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0873fff640 1 -- 172.21.15.40:0/2749672055 >> v1:172.21.15.40:6789/0 conn(0x7f08740b23d0 legacy=0x7f087409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f0873fff640 1 -- 172.21.15.40:0/2749672055 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08740b08a0 con 0x7f0874073230 2024-04-02T16:11:29.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.001+0000 7f08737fe640 1 -- 172.21.15.40:0/2749672055 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088220960 0 0) 0x7f08740b19e0 con 0x7f08740b23d0 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f0873fff640 1 --2- 172.21.15.40:0/2749672055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874073230 0x7f08740b27b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0874146560 tx=0x7f086801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf561750fd0d004 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08737fe640 1 -- 172.21.15.40:0/2749672055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680ab610 con 0x7f0874073230 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08737fe640 1 -- 172.21.15.40:0/2749672055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08680b0070 con 0x7f0874073230 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08737fe640 1 -- 172.21.15.40:0/2749672055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680ab9d0 con 0x7f0874073230 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/2749672055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874073230 msgr2=0x7f08740b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- 172.21.15.40:0/2749672055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874073230 0x7f08740b27b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0874146560 tx=0x7f086801c3f0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/2749672055 shutdown_connections 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- 172.21.15.40:0/2749672055 >> v2:172.21.15.204:3300/0 conn(0x7f0874148e50 0x7f0874149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- 172.21.15.40:0/2749672055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874073230 0x7f08740b27b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/2749672055 >> 172.21.15.40:0/2749672055 conn(0x7f08740b8fd0 msgr2=0x7f08740b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:29.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/2749672055 shutdown_connections 2024-04-02T16:11:29.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/2749672055 wait complete. 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 Processor -- start 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- start start 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0874073230 0x7f0874147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08740b23d0 0x7f08741e3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0874152810 con 0x7f0874148e50 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0874152690 con 0x7f0874073230 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0874152990 con 0x7f08740b23d0 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36140/0 (socket says 172.21.15.40:36140) 2024-04-02T16:11:29.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 -- 172.21.15.40:0/1679628910 learned_addr learned my addr 172.21.15.40:0/1679628910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f0878bde640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0874073230 0x7f0874147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f0873fff640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08740b23d0 0x7f08741e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 -- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08740b23d0 msgr2=0x7f08741e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08740b23d0 0x7f08741e3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 -- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0874073230 msgr2=0x7f0874147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0874073230 0x7f0874147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 -- 172.21.15.40:0/1679628910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08740aedb0 con 0x7f0874148e50 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08793df640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f086400d560 tx=0x7f086400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640125f0 con 0x7f0874148e50 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0864019070 con 0x7f0874148e50 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08740af090 con 0x7f0874148e50 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640129f0 con 0x7f0874148e50 2024-04-02T16:11:29.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.005+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08740af890 con 0x7f0874148e50 2024-04-02T16:11:29.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.009+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740685f0 con 0x7f0874148e50 2024-04-02T16:11:29.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.009+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0864012b90 con 0x7f0874148e50 2024-04-02T16:11:29.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.009+0000 7f08717fa640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 0x7f086006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.009+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f08640869a0 con 0x7f0874148e50 2024-04-02T16:11:29.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.009+0000 7f0878bde640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 0x7f086006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.013+0000 7f0878bde640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 0x7f086006a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f086c0012f0 tx=0x7f086c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.013+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f0864057340 con 0x7f0874148e50 2024-04-02T16:11:29.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.169+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0874146340 con 0x7f0874148e50 2024-04-02T16:11:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f08717fa640 1 -- 172.21.15.40:0/1679628910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f0864056c40 con 0x7f0874148e50 2024-04-02T16:11:29.176 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 msgr2=0x7f086006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 0x7f086006a870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f086c0012f0 tx=0x7f086c009040 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 msgr2=0x7f0874145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f086400d560 tx=0x7f086400da20 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 shutdown_connections 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f08740b23d0 0x7f08741e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f08600683b0 0x7f086006a870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0874148e50 0x7f0874145780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 --2- 172.21.15.40:0/1679628910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0874073230 0x7f0874147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 >> 172.21.15.40:0/1679628910 conn(0x7f08740b8fd0 msgr2=0x7f087406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 shutdown_connections 2024-04-02T16:11:29.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.173+0000 7f087ae42640 1 -- 172.21.15.40:0/1679628910 wait complete. 2024-04-02T16:11:29.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T16:11:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.297+0000 7ff0356b4640 1 Processor -- start 2024-04-02T16:11:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.297+0000 7ff0356b4640 1 -- start start 2024-04-02T16:11:29.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff030072a70 0x7ff0301455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff030145b00 0x7ff0300718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0300b3a30 con 0x7ff030072e50 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff0300b2360 con 0x7ff030145b00 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff0300b3da0 con 0x7ff030072a70 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff030145b00 0x7ff0300718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff030145b00 0x7ff0300718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36152/0 (socket says 172.21.15.40:36152) 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 -- 172.21.15.40:0/2239014443 learned_addr learned my addr 172.21.15.40:0/2239014443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 -- 172.21.15.40:0/2239014443 >> v2:172.21.15.204:3300/0 conn(0x7ff030072a70 msgr2=0x7ff0301455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 --2- 172.21.15.40:0/2239014443 >> v2:172.21.15.204:3300/0 conn(0x7ff030072a70 0x7ff0301455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 -- 172.21.15.40:0/2239014443 >> v1:172.21.15.204:6789/0 conn(0x7ff030072e50 legacy=0x7ff030071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 -- 172.21.15.40:0/2239014443 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0300b4230 con 0x7ff030145b00 2024-04-02T16:11:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02e7fc640 1 --2- 172.21.15.40:0/2239014443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff0300718d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0300682d0 tx=0x7ff02401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351dcd991ad37302 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02dffb640 1 -- 172.21.15.40:0/2239014443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240b0070 con 0x7ff030145b00 2024-04-02T16:11:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02dffb640 1 -- 172.21.15.40:0/2239014443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0240ab600 con 0x7ff030145b00 2024-04-02T16:11:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff02dffb640 1 -- 172.21.15.40:0/2239014443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240ab9d0 con 0x7ff030145b00 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- 172.21.15.40:0/2239014443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 msgr2=0x7ff0300718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 --2- 172.21.15.40:0/2239014443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff0300718d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0300682d0 tx=0x7ff02401c4e0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- 172.21.15.40:0/2239014443 shutdown_connections 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 --2- 172.21.15.40:0/2239014443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff0300718d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 --2- 172.21.15.40:0/2239014443 >> v2:172.21.15.204:3300/0 conn(0x7ff030072a70 0x7ff0301455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- 172.21.15.40:0/2239014443 >> 172.21.15.40:0/2239014443 conn(0x7ff0300b7d10 msgr2=0x7ff0300b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- 172.21.15.40:0/2239014443 shutdown_connections 2024-04-02T16:11:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.301+0000 7ff0356b4640 1 -- 172.21.15.40:0/2239014443 wait complete. 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 Processor -- start 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- start start 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff030072a70 0x7ff030142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff030072e50 0x7ff0301ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301528d0 con 0x7ff030145b00 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0300734e0 con 0x7ff030072e50 2024-04-02T16:11:29.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff030152a50 con 0x7ff030072a70 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36166/0 (socket says 172.21.15.40:36166) 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 -- 172.21.15.40:0/720549252 learned_addr learned my addr 172.21.15.40:0/720549252 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 -- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff030072a70 msgr2=0x7ff030142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff030072a70 0x7ff030142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 -- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff030072e50 msgr2=0x7ff0301ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff030072e50 0x7ff0301ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 -- 172.21.15.40:0/720549252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03009cc70 con 0x7ff030145b00 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff02f7fe640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c410 tx=0x7ff02800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028017070 con 0x7ff030145b00 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff028003600 con 0x7ff030145b00 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03009f580 con 0x7ff030145b00 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028003a30 con 0x7ff030145b00 2024-04-02T16:11:29.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03009b510 con 0x7ff030145b00 2024-04-02T16:11:29.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.305+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff030068d00 con 0x7ff030145b00 2024-04-02T16:11:29.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff02801a070 con 0x7ff030145b00 2024-04-02T16:11:29.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff0177fe640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff028085de0 con 0x7ff030145b00 2024-04-02T16:11:29.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff02effd640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff02effd640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 0x7ff01c06a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ab90 tx=0x7ff020009250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.309+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff028021020 con 0x7ff030145b00 2024-04-02T16:11:29.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.473+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff030146130 con 0x7ff030145b00 2024-04-02T16:11:29.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.473+0000 7ff0177fe640 1 -- 172.21.15.40:0/720549252 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7ff028056860 con 0x7ff030145b00 2024-04-02T16:11:29.481 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:11:29.482 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-02T16:11:29.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 msgr2=0x7ff01c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 0x7ff01c06a840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ab90 tx=0x7ff020009250 comp rx=0 tx=0).stop 2024-04-02T16:11:29.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 msgr2=0x7ff03009e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff02800c410 tx=0x7ff02800c8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 shutdown_connections 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff01c068380 0x7ff01c06a840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff030145b00 0x7ff03009e6b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff030072e50 0x7ff0301ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 --2- 172.21.15.40:0/720549252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff030072a70 0x7ff030142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 >> 172.21.15.40:0/720549252 conn(0x7ff0300b7d10 msgr2=0x7ff0300b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 shutdown_connections 2024-04-02T16:11:29.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.477+0000 7ff0356b4640 1 -- 172.21.15.40:0/720549252 wait complete. 2024-04-02T16:11:29.497 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T16:11:29.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T16:11:29.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.621+0000 7f6a9007f640 1 Processor -- start 2024-04-02T16:11:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.621+0000 7f6a9007f640 1 -- start start 2024-04-02T16:11:29.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a9007f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 0x7f6a880b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a9007f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8806e450 con 0x7f6a88150060 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a9007f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a880b59b0 con 0x7f6a880b74f0 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a9007f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880b5c90 con 0x7f6a88148380 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6a88150060 0x7f6a88152450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49174/0 (socket says 172.21.15.40:49174) 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 -- 172.21.15.40:0/1386620541 learned_addr learned my addr 172.21.15.40:0/1386620541 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8de1b640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 0x7f6a880b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555192959 0 0) 0x7f6a8806e450 con 0x7f6a88150060 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a70003530 con 0x7f6a88150060 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1502383317 0 0) 0x7f6a70003530 con 0x7f6a88150060 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 msgr2=0x7f6a880b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 0x7f6a880b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 >> v1:172.21.15.204:6789/0 conn(0x7f6a880b74f0 legacy=0x7f6a8806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880b5f70 con 0x7f6a88150060 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f6a78001680 con 0x7f6a88150060 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 >> v1:172.21.15.40:6789/0 conn(0x7f6a88150060 legacy=0x7f6a88152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a700039f0 0x7f6a70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 0x7f6a70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a70004d50 0x7f6a700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880b5c90 con 0x7f6a70004d50 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7000bb00 con 0x7f6a700039f0 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7000bd00 con 0x7f6a88148380 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a70004d50 0x7f6a700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 -- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 msgr2=0x7f6a70004810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a88148380 0x7f6a70004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 -- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a700039f0 msgr2=0x7f6a70003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:11:29.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a700039f0 0x7f6a70003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.625+0000 7f6a8e61c640 1 -- 172.21.15.40:0/1386620541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a70003530 con 0x7f6a70004d50 2024-04-02T16:11:29.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a70004d50 0x7f6a700051d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a88071840 tx=0x7f6a7801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57532bc6cd9e74f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007d20 con 0x7f6a70004d50 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a78005ae0 con 0x7f6a70004d50 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8ce19640 1 -- 172.21.15.40:0/1386620541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007070 con 0x7f6a70004d50 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/1386620541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a70004d50 msgr2=0x7f6a700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 --2- 172.21.15.40:0/1386620541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a70004d50 0x7f6a700051d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6a88071840 tx=0x7f6a7801f390 comp rx=0 tx=0).stop 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8e61c640 1 -- 172.21.15.40:0/1386620541 reap_dead start 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/1386620541 shutdown_connections 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/1386620541 >> 172.21.15.40:0/1386620541 conn(0x7f6a880bb690 msgr2=0x7f6a880b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/1386620541 shutdown_connections 2024-04-02T16:11:29.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/1386620541 wait complete. 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 Processor -- start 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- start start 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 0x7f6a881431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 0x7f6a8813e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88147770 con 0x7f6a88144040 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88073810 con 0x7f6a881e0e70 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6a881478f0 con 0x7f6a70004d50 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36176/0 (socket says 172.21.15.40:36176) 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 -- 172.21.15.40:0/2940411530 learned_addr learned my addr 172.21.15.40:0/2940411530 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8de1b640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 0x7f6a881431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 0x7f6a8813e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 -- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 msgr2=0x7f6a881431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 0x7f6a881431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 -- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 msgr2=0x7f6a8813e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 0x7f6a8813e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 -- 172.21.15.40:0/2940411530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a88144520 con 0x7f6a88144040 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8e61c640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 0x7f6a8813e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:29.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8de1b640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 0x7f6a881431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a8d61a640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c460 tx=0x7f6a7c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c017070 con 0x7f6a88144040 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a7c0052d0 con 0x7f6a88144040 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a880b4450 con 0x7f6a88144040 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003840 con 0x7f6a88144040 2024-04-02T16:11:29.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.629+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8813ede0 con 0x7f6a88144040 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a880b4270 con 0x7f6a88144040 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6a7c01a070 con 0x7f6a88144040 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a867fc640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 0x7f6a7406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6a7c085950 con 0x7f6a88144040 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a8de1b640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 0x7f6a7406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:29.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.633+0000 7f6a8de1b640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 0x7f6a7406a870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a80007520 tx=0x7f6a8000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:29.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.637+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f6a880b4270 con 0x7f6a88144040 2024-04-02T16:11:29.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:29.797+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a88068d00 con 0x7f6a88144040 2024-04-02T16:11:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.085+0000 7f6a867fc640 1 -- 172.21.15.40:0/2940411530 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f6a7c005480 con 0x7f6a88144040 2024-04-02T16:11:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 msgr2=0x7f6a7406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 0x7f6a7406a870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a80007520 tx=0x7f6a8000b040 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 msgr2=0x7f6a881e0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c460 tx=0x7f6a7c00c920 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 shutdown_connections 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6a740683b0 0x7f6a7406a870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a881e0e70 0x7f6a8813e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a88144040 0x7f6a881e0930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 --2- 172.21.15.40:0/2940411530 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a70004d50 0x7f6a881431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 >> 172.21.15.40:0/2940411530 conn(0x7f6a880bb690 msgr2=0x7f6a881465b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 shutdown_connections 2024-04-02T16:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.089+0000 7f6a9007f640 1 -- 172.21.15.40:0/2940411530 wait complete. 2024-04-02T16:11:30.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T16:11:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 Processor -- start 2024-04-02T16:11:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 -- start start 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38ac0b23d0 0x7f38ac0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 0x7f38ac149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38ac0b19e0 con 0x7f38ac073230 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f38ac1497c0 con 0x7f38ac0b23d0 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b2331640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f38ac0b05c0 con 0x7f38ac148e50 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38abfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38ac0b23d0 0x7f38ac0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38ab7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f38ac073230 0x7f38ac09daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49180/0 (socket says 172.21.15.40:49180) 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38ab7fe640 1 -- 172.21.15.40:0/2447501720 learned_addr learned my addr 172.21.15.40:0/2447501720 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b08ce640 1 --2- 172.21.15.40:0/2447501720 >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 0x7f38ac149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38aaffd640 1 -- 172.21.15.40:0/2447501720 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3250854057 0 0) 0x7f38ac0b19e0 con 0x7f38ac073230 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38aaffd640 1 -- 172.21.15.40:0/2447501720 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3894003530 con 0x7f38ac073230 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38abfff640 1 -- 172.21.15.40:0/2447501720 >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 msgr2=0x7f38ac149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38abfff640 1 --2- 172.21.15.40:0/2447501720 >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 0x7f38ac149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38abfff640 1 -- 172.21.15.40:0/2447501720 >> v1:172.21.15.40:6789/0 conn(0x7f38ac073230 legacy=0x7f38ac09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38abfff640 1 -- 172.21.15.40:0/2447501720 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac0b08a0 con 0x7f38ac0b23d0 2024-04-02T16:11:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.249+0000 7f38b08ce640 1 --2- 172.21.15.40:0/2447501720 >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 0x7f38ac149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38abfff640 1 --2- 172.21.15.40:0/2447501720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0b27b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38ac146560 tx=0x7f38a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c68c2b4f4d9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38aaffd640 1 -- 172.21.15.40:0/2447501720 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00ab610 con 0x7f38ac0b23d0 2024-04-02T16:11:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38aaffd640 1 -- 172.21.15.40:0/2447501720 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a00b0070 con 0x7f38ac0b23d0 2024-04-02T16:11:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38aaffd640 1 -- 172.21.15.40:0/2447501720 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00ab9d0 con 0x7f38ac0b23d0 2024-04-02T16:11:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 -- 172.21.15.40:0/2447501720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac0b23d0 msgr2=0x7f38ac0b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 --2- 172.21.15.40:0/2447501720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0b27b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38ac146560 tx=0x7f38a001c3f0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 -- 172.21.15.40:0/2447501720 shutdown_connections 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 --2- 172.21.15.40:0/2447501720 >> v2:172.21.15.204:3300/0 conn(0x7f38ac148e50 0x7f38ac149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 --2- 172.21.15.40:0/2447501720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0b27b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 -- 172.21.15.40:0/2447501720 >> 172.21.15.40:0/2447501720 conn(0x7f38ac0b8fd0 msgr2=0x7f38ac0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 -- 172.21.15.40:0/2447501720 shutdown_connections 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.253+0000 7f38b2331640 1 -- 172.21.15.40:0/2447501720 wait complete. 2024-04-02T16:11:30.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 Processor -- start 2024-04-02T16:11:30.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- start start 2024-04-02T16:11:30.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38ac148e50 0x7f38ac136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac152810 con 0x7f38ac073230 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac152690 con 0x7f38ac0b23d0 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac152990 con 0x7f38ac148e50 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36184/0 (socket says 172.21.15.40:36184) 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 -- 172.21.15.40:0/862943667 learned_addr learned my addr 172.21.15.40:0/862943667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b08ce640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38ac148e50 0x7f38ac136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38ab7fe640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 -- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38ac148e50 msgr2=0x7f38ac136f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38ac148e50 0x7f38ac136f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 -- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ac0b23d0 msgr2=0x7f38ac0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 -- 172.21.15.40:0/862943667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac147100 con 0x7f38ac073230 2024-04-02T16:11:30.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38abfff640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38a00ad040 tx=0x7f38a001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00bf410 con 0x7f38ac073230 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a00bf5b0 con 0x7f38ac073230 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac1473e0 con 0x7f38ac073230 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00bf9a0 con 0x7f38ac073230 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac137c90 con 0x7f38ac073230 2024-04-02T16:11:30.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.257+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac0685f0 con 0x7f38ac073230 2024-04-02T16:11:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.261+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f38a00bd020 con 0x7f38ac073230 2024-04-02T16:11:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.261+0000 7f38a8ff9640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 0x7f389806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.261+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f38a0120ef0 con 0x7f38ac073230 2024-04-02T16:11:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.261+0000 7f38ab7fe640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 0x7f389806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.261+0000 7f38ab7fe640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 0x7f389806a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38ac1de0b0 tx=0x7f389c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.265+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f38a00a8050 con 0x7f38ac073230 2024-04-02T16:11:30.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:30.429+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f38ac0b56e0 con 0x7f38ac073230 2024-04-02T16:11:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.089+0000 7f38a8ff9640 1 -- 172.21.15.40:0/862943667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f38a0016070 con 0x7f38ac073230 2024-04-02T16:11:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 msgr2=0x7f389806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 0x7f389806a840 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38ac1de0b0 tx=0x7f389c009040 comp rx=0 tx=0).stop 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 msgr2=0x7f38ac138e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38a00ad040 tx=0x7f38a001ac30 comp rx=0 tx=0).stop 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 shutdown_connections 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38ac148e50 0x7f38ac136f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3898068380 0x7f389806a840 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ac0b23d0 0x7f38ac0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 --2- 172.21.15.40:0/862943667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38ac073230 0x7f38ac138e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 >> 172.21.15.40:0/862943667 conn(0x7f38ac0b8fd0 msgr2=0x7f38ac06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 shutdown_connections 2024-04-02T16:11:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.093+0000 7f38b2331640 1 -- 172.21.15.40:0/862943667 wait complete. 2024-04-02T16:11:31.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T16:11:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 Processor -- start 2024-04-02T16:11:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 -- start start 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb768147c50 0x7fb7681480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb768148620 0x7fb76814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb768148a90 con 0x7fb76806c8a0 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7680b2b50 con 0x7fb768147c50 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76d170640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb7680b0300 con 0x7fb768148620 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb768147c50 0x7fb7681480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb766d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb76806c8a0 0x7fb7680af150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49188/0 (socket says 172.21.15.40:49188) 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb766d9d640 1 -- 172.21.15.40:0/1600401084 learned_addr learned my addr 172.21.15.40:0/1600401084 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:31.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76759e640 1 --2- 172.21.15.40:0/1600401084 >> v2:172.21.15.204:3300/0 conn(0x7fb768148620 0x7fb76814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76659c640 1 -- 172.21.15.40:0/1600401084 >> v2:172.21.15.204:3300/0 conn(0x7fb768148620 msgr2=0x7fb76814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76659c640 1 --2- 172.21.15.40:0/1600401084 >> v2:172.21.15.204:3300/0 conn(0x7fb768148620 0x7fb76814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76659c640 1 -- 172.21.15.40:0/1600401084 >> v1:172.21.15.40:6789/0 conn(0x7fb76806c8a0 legacy=0x7fb7680af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.257+0000 7fb76659c640 1 -- 172.21.15.40:0/1600401084 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7680ae830 con 0x7fb768147c50 2024-04-02T16:11:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76659c640 1 --2- 172.21.15.40:0/1600401084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb768147c50 0x7fb7681480e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb768072f90 tx=0x7fb75c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d5519e5a6e6ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb765d9b640 1 -- 172.21.15.40:0/1600401084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b0070 con 0x7fb768147c50 2024-04-02T16:11:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb765d9b640 1 -- 172.21.15.40:0/1600401084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0ab600 con 0x7fb768147c50 2024-04-02T16:11:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb765d9b640 1 -- 172.21.15.40:0/1600401084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0ab9d0 con 0x7fb768147c50 2024-04-02T16:11:31.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/1600401084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb768147c50 msgr2=0x7fb7681480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- 172.21.15.40:0/1600401084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb768147c50 0x7fb7681480e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb768072f90 tx=0x7fb75c01c3f0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/1600401084 shutdown_connections 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- 172.21.15.40:0/1600401084 >> v2:172.21.15.204:3300/0 conn(0x7fb768148620 0x7fb76814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- 172.21.15.40:0/1600401084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb768147c50 0x7fb7681480e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb768072f90 tx=0x7fb75c01c3f0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/1600401084 >> 172.21.15.40:0/1600401084 conn(0x7fb7680b8fd0 msgr2=0x7fb7680b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/1600401084 shutdown_connections 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/1600401084 wait complete. 2024-04-02T16:11:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 Processor -- start 2024-04-02T16:11:31.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- start start 2024-04-02T16:11:31.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb768148620 0x7fb76809c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7681e3be0 0x7fb7681e4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680b3130 con 0x7fb76806c8a0 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680b0540 con 0x7fb768148620 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb7681e4560 con 0x7fb7681e3be0 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36210/0 (socket says 172.21.15.40:36210) 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 -- 172.21.15.40:0/345846269 learned_addr learned my addr 172.21.15.40:0/345846269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76659c640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb768148620 0x7fb76809c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76759e640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7681e3be0 0x7fb7681e4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 -- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7681e3be0 msgr2=0x7fb7681e4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7681e3be0 0x7fb7681e4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 -- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb768148620 msgr2=0x7fb76809c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:31.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb768148620 0x7fb76809c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 -- 172.21.15.40:0/345846269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76809d500 con 0x7fb76806c8a0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb766d9d640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb75800cb10 tx=0x7fb75800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758017070 con 0x7fb76806c8a0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75800ac70 con 0x7fb76806c8a0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb768146800 con 0x7fb76806c8a0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758012620 con 0x7fb76806c8a0 2024-04-02T16:11:31.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.261+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76809b390 con 0x7fb76806c8a0 2024-04-02T16:11:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7680685f0 con 0x7fb76806c8a0 2024-04-02T16:11:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb75801a070 con 0x7fb76806c8a0 2024-04-02T16:11:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb74f7fe640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 0x7fb75406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fb758086b20 con 0x7fb76806c8a0 2024-04-02T16:11:31.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb76659c640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 0x7fb75406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.265+0000 7fb76659c640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 0x7fb75406a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ad000 tx=0x7fb75c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:31.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.269+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb758021020 con 0x7fb76806c8a0 2024-04-02T16:11:31.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:31.465+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb7680b32b0 con 0x7fb76806c8a0 2024-04-02T16:11:31.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:11:31.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:11:32.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.145+0000 7fb74f7fe640 1 -- 172.21.15.40:0/345846269 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fb7580571b0 con 0x7fb76806c8a0 2024-04-02T16:11:32.154 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T16:11:32.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 msgr2=0x7fb75406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 0x7fb75406a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ad000 tx=0x7fb75c0a6000 comp rx=0 tx=0).stop 2024-04-02T16:11:32.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 msgr2=0x7fb76809cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb75800cb10 tx=0x7fb75800a060 comp rx=0 tx=0).stop 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 shutdown_connections 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7681e3be0 0x7fb7681e4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb754068380 0x7fb75406a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb768148620 0x7fb76809c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 --2- 172.21.15.40:0/345846269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb76806c8a0 0x7fb76809cea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.157+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 >> 172.21.15.40:0/345846269 conn(0x7fb7680b8fd0 msgr2=0x7fb76806ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.161+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 shutdown_connections 2024-04-02T16:11:32.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.161+0000 7fb76d170640 1 -- 172.21.15.40:0/345846269 wait complete. 2024-04-02T16:11:32.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 Processor -- start 2024-04-02T16:11:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 -- start start 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28f40b23d0 0x7f28f40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28f4148e50 0x7f28f4149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28f40b19e0 con 0x7f28f4073230 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f28f41497c0 con 0x7f28f4148e50 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28fb568640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f28f40b05c0 con 0x7f28f40b23d0 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28f4148e50 0x7f28f4149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f28f4148e50 0x7f28f4149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36212/0 (socket says 172.21.15.40:36212) 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 -- 172.21.15.40:0/1674332998 learned_addr learned my addr 172.21.15.40:0/1674332998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9304640 1 --2- 172.21.15.40:0/1674332998 >> v2:172.21.15.204:3300/0 conn(0x7f28f40b23d0 0x7f28f40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 -- 172.21.15.40:0/1674332998 >> v2:172.21.15.204:3300/0 conn(0x7f28f40b23d0 msgr2=0x7f28f40b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 --2- 172.21.15.40:0/1674332998 >> v2:172.21.15.204:3300/0 conn(0x7f28f40b23d0 0x7f28f40b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 -- 172.21.15.40:0/1674332998 >> v1:172.21.15.204:6789/0 conn(0x7f28f4073230 legacy=0x7f28f409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.309+0000 7f28f9b05640 1 -- 172.21.15.40:0/1674332998 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f40b08a0 con 0x7f28f4148e50 2024-04-02T16:11:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28f9b05640 1 --2- 172.21.15.40:0/1674332998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f4148e50 0x7f28f4149280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28f40b00c0 tx=0x7f28ec01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65167d33babb3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28ebfff640 1 -- 172.21.15.40:0/1674332998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0a5800 con 0x7f28f4148e50 2024-04-02T16:11:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28ebfff640 1 -- 172.21.15.40:0/1674332998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28ec0a59a0 con 0x7f28f4148e50 2024-04-02T16:11:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28ebfff640 1 -- 172.21.15.40:0/1674332998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0a5d70 con 0x7f28f4148e50 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- 172.21.15.40:0/1674332998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f4148e50 msgr2=0x7f28f4149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 --2- 172.21.15.40:0/1674332998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f4148e50 0x7f28f4149280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28f40b00c0 tx=0x7f28ec01c920 comp rx=0 tx=0).stop 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- 172.21.15.40:0/1674332998 shutdown_connections 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 --2- 172.21.15.40:0/1674332998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f4148e50 0x7f28f4149280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 --2- 172.21.15.40:0/1674332998 >> v2:172.21.15.204:3300/0 conn(0x7f28f40b23d0 0x7f28f40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- 172.21.15.40:0/1674332998 >> 172.21.15.40:0/1674332998 conn(0x7f28f40b8fd0 msgr2=0x7f28f40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- 172.21.15.40:0/1674332998 shutdown_connections 2024-04-02T16:11:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- 172.21.15.40:0/1674332998 wait complete. 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 Processor -- start 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.313+0000 7f28fb568640 1 -- start start 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 0x7f28f413c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 0x7f28f41e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4152810 con 0x7f28f40b23d0 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4152690 con 0x7f28f4073230 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28f4152990 con 0x7f28f4148e50 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36214/0 (socket says 172.21.15.40:36214) 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 -- 172.21.15.40:0/405141831 learned_addr learned my addr 172.21.15.40:0/405141831 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f9304640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 0x7f28f413c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f9b05640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 0x7f28f41e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 -- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 msgr2=0x7f28f41e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 0x7f28f41e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 -- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 msgr2=0x7f28f413c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 0x7f28f413c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 -- 172.21.15.40:0/405141831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f41438a0 con 0x7f28f40b23d0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f9b05640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 0x7f28f41e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f9304640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 0x7f28f413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f8b03640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c480 tx=0x7f28e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012830 con 0x7f28f40b23d0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28e4017070 con 0x7f28f40b23d0 2024-04-02T16:11:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f4145df0 con 0x7f28f40b23d0 2024-04-02T16:11:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012c30 con 0x7f28f40b23d0 2024-04-02T16:11:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f4143b20 con 0x7f28f40b23d0 2024-04-02T16:11:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f40685f0 con 0x7f28f40b23d0 2024-04-02T16:11:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f28e401c460 con 0x7f28f40b23d0 2024-04-02T16:11:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 0x7f28d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f28e40867a0 con 0x7f28f40b23d0 2024-04-02T16:11:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.317+0000 7f28f9304640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 0x7f28d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.321+0000 7f28f9304640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 0x7f28d806a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009840 tx=0x7f28f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.321+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f28f4145df0 con 0x7f28f40b23d0 2024-04-02T16:11:32.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.485+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28f40b0300 con 0x7f28f40b23d0 2024-04-02T16:11:32.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28e9ffb640 1 -- 172.21.15.40:0/405141831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f28f41438a0 con 0x7f28f40b23d0 2024-04-02T16:11:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:32.493 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:31.472871+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 msgr2=0x7f28d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 0x7f28d806a840 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009840 tx=0x7f28f0008040 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 msgr2=0x7f28f41458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28e400c480 tx=0x7f28e400c940 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 shutdown_connections 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28f4148e50 0x7f28f41e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f28d8068380 0x7f28d806a840 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28f40b23d0 0x7f28f41458b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 --2- 172.21.15.40:0/405141831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28f4073230 0x7f28f413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 >> 172.21.15.40:0/405141831 conn(0x7f28f40b8fd0 msgr2=0x7f28f40b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 shutdown_connections 2024-04-02T16:11:32.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.489+0000 7f28fb568640 1 -- 172.21.15.40:0/405141831 wait complete. 2024-04-02T16:11:32.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 Processor -- start 2024-04-02T16:11:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 -- start start 2024-04-02T16:11:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c40b23d0 0x7f91c40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c4073230 0x7f91c409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f91c409dfe0 0x7f91c4148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f91c40b19e0 con 0x7f91c4073230 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f91c40b05c0 con 0x7f91c409dfe0 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91caedd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f91c40b08a0 con 0x7f91c40b23d0 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c4073230 0x7f91c409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91c4073230 0x7f91c409daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36228/0 (socket says 172.21.15.40:36228) 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 -- 172.21.15.40:0/2151871001 learned_addr learned my addr 172.21.15.40:0/2151871001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c8c79640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c40b23d0 0x7f91c40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c947a640 1 --2- 172.21.15.40:0/2151871001 >> v2:172.21.15.204:3300/0 conn(0x7f91c409dfe0 0x7f91c4148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 -- 172.21.15.40:0/2151871001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c40b23d0 msgr2=0x7f91c40b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c40b23d0 0x7f91c40b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 -- 172.21.15.40:0/2151871001 >> v2:172.21.15.204:3300/0 conn(0x7f91c409dfe0 msgr2=0x7f91c4148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 --2- 172.21.15.40:0/2151871001 >> v2:172.21.15.204:3300/0 conn(0x7f91c409dfe0 0x7f91c4148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.617+0000 7f91c3fff640 1 -- 172.21.15.40:0/2151871001 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c40b0b80 con 0x7f91c4073230 2024-04-02T16:11:32.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c3fff640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c4073230 0x7f91c409daa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91c406be30 tx=0x7f91b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81643881b01edec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c37fe640 1 -- 172.21.15.40:0/2151871001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80b0070 con 0x7f91c4073230 2024-04-02T16:11:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c37fe640 1 -- 172.21.15.40:0/2151871001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b80ab600 con 0x7f91c4073230 2024-04-02T16:11:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c37fe640 1 -- 172.21.15.40:0/2151871001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80ab9d0 con 0x7f91c4073230 2024-04-02T16:11:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/2151871001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c4073230 msgr2=0x7f91c409daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c4073230 0x7f91c409daa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91c406be30 tx=0x7f91b801c490 comp rx=0 tx=0).stop 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/2151871001 shutdown_connections 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- 172.21.15.40:0/2151871001 >> v2:172.21.15.204:3300/0 conn(0x7f91c409dfe0 0x7f91c4148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c4073230 0x7f91c409daa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- 172.21.15.40:0/2151871001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c40b23d0 0x7f91c40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/2151871001 >> 172.21.15.40:0/2151871001 conn(0x7f91c40b8fd0 msgr2=0x7f91c40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/2151871001 shutdown_connections 2024-04-02T16:11:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/2151871001 wait complete. 2024-04-02T16:11:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 Processor -- start 2024-04-02T16:11:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- start start 2024-04-02T16:11:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c4073230 0x7f91c413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 0x7f91c41e2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4146130 con 0x7f91c40b23d0 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4145fb0 con 0x7f91c409dfe0 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f91c41462b0 con 0x7f91c4073230 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36232/0 (socket says 172.21.15.40:36232) 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 -- 172.21.15.40:0/3744172991 learned_addr learned my addr 172.21.15.40:0/3744172991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c8c79640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c4073230 0x7f91c413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c3fff640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 0x7f91c41e2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 -- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c4073230 msgr2=0x7f91c413b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c4073230 0x7f91c413b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 -- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 msgr2=0x7f91c41e2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 0x7f91c41e2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 -- 172.21.15.40:0/3744172991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c4147ef0 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c3fff640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 0x7f91c41e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c947a640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91bc007e40 tx=0x7f91bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc014620 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc019070 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c41481d0 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc014a20 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.621+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c413b7a0 con 0x7f91c40b23d0 2024-04-02T16:11:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c40b3470 con 0x7f91c40b23d0 2024-04-02T16:11:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f91bc014bc0 con 0x7f91c40b23d0 2024-04-02T16:11:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91c17fa640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 0x7f91b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f91bc087260 con 0x7f91c40b23d0 2024-04-02T16:11:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91c8c79640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 0x7f91b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.625+0000 7f91c8c79640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 0x7f91b006a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91b400b950 tx=0x7f91b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.629+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f91bc057820 con 0x7f91c40b23d0 2024-04-02T16:11:32.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.789+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91c40685f0 con 0x7f91c40b23d0 2024-04-02T16:11:32.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91c17fa640 1 -- 172.21.15.40:0/3744172991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16570 (secure 0 0 0) 0x7f91bc057120 con 0x7f91c40b23d0 2024-04-02T16:11:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:32.796 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:31.472871+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 msgr2=0x7f91b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 0x7f91b006a840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f91b400b950 tx=0x7f91b4008040 comp rx=0 tx=0).stop 2024-04-02T16:11:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 msgr2=0x7f91c41e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91bc007e40 tx=0x7f91bc00e4b0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 shutdown_connections 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f91b0068380 0x7f91b006a840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91c40b23d0 0x7f91c41e2670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91c409dfe0 0x7f91c41e2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 --2- 172.21.15.40:0/3744172991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91c4073230 0x7f91c413b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 >> 172.21.15.40:0/3744172991 conn(0x7f91c40b8fd0 msgr2=0x7f91c40b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 shutdown_connections 2024-04-02T16:11:32.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.793+0000 7f91caedd640 1 -- 172.21.15.40:0/3744172991 wait complete. 2024-04-02T16:11:32.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T16:11:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 Processor -- start 2024-04-02T16:11:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 -- start start 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faa980b5a00 0x7faa9814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa98152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980b4c20 con 0x7faa9814b240 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faa980b44b0 con 0x7faa980b5a00 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa9e218640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faa980b6f10 con 0x7faa980ba020 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa96ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa9814b240 0x7faa98141200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49190/0 (socket says 172.21.15.40:49190) 2024-04-02T16:11:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa96ffd640 1 -- 172.21.15.40:0/2399862773 learned_addr learned my addr 172.21.15.40:0/2399862773 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa98152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa977fe640 1 --2- 172.21.15.40:0/2399862773 >> v2:172.21.15.204:3300/0 conn(0x7faa980b5a00 0x7faa9814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819986457 0 0) 0x7faa980b4c20 con 0x7faa9814b240 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa80003530 con 0x7faa9814b240 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 821243481 0 0) 0x7faa80003530 con 0x7faa9814b240 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 msgr2=0x7faa98152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa98152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 >> v2:172.21.15.204:3300/0 conn(0x7faa980b5a00 msgr2=0x7faa9814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 --2- 172.21.15.40:0/2399862773 >> v2:172.21.15.204:3300/0 conn(0x7faa980b5a00 0x7faa9814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980ba540 con 0x7faa9814b240 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7faa8c001680 con 0x7faa9814b240 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 >> v1:172.21.15.40:6789/0 conn(0x7faa9814b240 legacy=0x7faa98141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa800039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa80003f90 0x7faa80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa80004d50 0x7faa800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b44b0 con 0x7faa80004d50 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b6f10 con 0x7faa80003f90 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faa8000bb00 con 0x7faa980ba020 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa80004d50 0x7faa800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa96ffd640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa80003f90 0x7faa80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa977fe640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa800039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 -- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 msgr2=0x7faa800039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa980ba020 0x7faa800039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 -- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa80003f90 msgr2=0x7faa80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa80003f90 0x7faa80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.941+0000 7faa97fff640 1 -- 172.21.15.40:0/2399862773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa80003530 con 0x7faa80004d50 2024-04-02T16:11:32.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa80004d50 0x7faa800051d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa980b92e0 tx=0x7faa9001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c54a4b682516b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900b2070 con 0x7faa80004d50 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa900a55b0 con 0x7faa80004d50 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa967fc640 1 -- 172.21.15.40:0/2399862773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900096b0 con 0x7faa80004d50 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- 172.21.15.40:0/2399862773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa80004d50 msgr2=0x7faa800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 --2- 172.21.15.40:0/2399862773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa80004d50 0x7faa800051d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa980b92e0 tx=0x7faa9001c9e0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 -- 172.21.15.40:0/2399862773 reap_dead start 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- 172.21.15.40:0/2399862773 shutdown_connections 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- 172.21.15.40:0/2399862773 >> 172.21.15.40:0/2399862773 conn(0x7faa980730b0 msgr2=0x7faa980734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- 172.21.15.40:0/2399862773 shutdown_connections 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- 172.21.15.40:0/2399862773 wait complete. 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 Processor -- start 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- start start 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 0x7faa981e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa981e37c0 0x7faa98141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b6940 con 0x7faa981423e0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b67c0 con 0x7faa981e37c0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa9e218640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faa981528a0 con 0x7faa98147ed0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36254/0 (socket says 172.21.15.40:36254) 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 -- 172.21.15.40:0/3313383336 learned_addr learned my addr 172.21.15.40:0/3313383336 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa977fe640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 0x7faa981e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa96ffd640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa981e37c0 0x7faa98141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 -- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 msgr2=0x7faa981e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 0x7faa981e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 -- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa981e37c0 msgr2=0x7faa98141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:32.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa981e37c0 0x7faa98141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa97fff640 1 -- 172.21.15.40:0/3313383336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981482b0 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.945+0000 7faa977fe640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 0x7faa981e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa97fff640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa9001d280 tx=0x7faa80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900b2040 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa900bb070 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa98143eb0 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900096b0 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa98144190 con 0x7faa981423e0 2024-04-02T16:11:32.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa980685f0 con 0x7faa981423e0 2024-04-02T16:11:32.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faa900c0460 con 0x7faa981423e0 2024-04-02T16:11:32.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 0x7faa8406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:32.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7faa901206b0 con 0x7faa981423e0 2024-04-02T16:11:32.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa977fe640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 0x7faa8406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:32.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.949+0000 7faa977fe640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 0x7faa8406a870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88007460 tx=0x7faa8800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:32.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:32.953+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7faa90016720 con 0x7faa981423e0 2024-04-02T16:11:33.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:33.149+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faa980bbea0 con 0x7faa981423e0 2024-04-02T16:11:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.165+0000 7faa7ffff640 1 -- 172.21.15.40:0/3313383336 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7faa900f0c70 con 0x7faa981423e0 2024-04-02T16:11:34.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 msgr2=0x7faa8406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 0x7faa8406a870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa88007460 tx=0x7faa8800b040 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 msgr2=0x7faa98142890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa9001d280 tx=0x7faa80005840 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 shutdown_connections 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faa840683b0 0x7faa8406a870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa981423e0 0x7faa98142890 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa981e37c0 0x7faa98141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 --2- 172.21.15.40:0/3313383336 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa98147ed0 0x7faa981e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 >> 172.21.15.40:0/3313383336 conn(0x7faa980730b0 msgr2=0x7faa980bc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 shutdown_connections 2024-04-02T16:11:34.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.169+0000 7faa9e218640 1 -- 172.21.15.40:0/3313383336 wait complete. 2024-04-02T16:11:34.199 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-02T16:11:34.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-02T16:11:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 Processor -- start 2024-04-02T16:11:34.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 -- start start 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc0b23d0 0x7f3efc0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3efc148e50 0x7f3efc1492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3efc0b19e0 con 0x7f3efc073230 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3efc149800 con 0x7f3efc148e50 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f03f3e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc0b05c0 con 0x7f3efc0b23d0 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3efc148e50 0x7f3efc1492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3efc148e50 0x7f3efc1492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36264/0 (socket says 172.21.15.40:36264) 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 -- 172.21.15.40:0/1493130856 learned_addr learned my addr 172.21.15.40:0/1493130856 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f01cda640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc0b23d0 0x7f3efc0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 -- 172.21.15.40:0/1493130856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc0b23d0 msgr2=0x7f3efc0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc0b23d0 0x7f3efc0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 -- 172.21.15.40:0/1493130856 >> v1:172.21.15.40:6789/0 conn(0x7f3efc073230 legacy=0x7f3efc09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.329+0000 7f3f024db640 1 -- 172.21.15.40:0/1493130856 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc0b08a0 con 0x7f3efc148e50 2024-04-02T16:11:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f024db640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc148e50 0x7f3efc1492c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5aa8c52095af9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f00cd8640 1 -- 172.21.15.40:0/1493130856 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40aa650 con 0x7f3efc148e50 2024-04-02T16:11:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f00cd8640 1 -- 172.21.15.40:0/1493130856 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef40af070 con 0x7f3efc148e50 2024-04-02T16:11:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f00cd8640 1 -- 172.21.15.40:0/1493130856 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40aaa10 con 0x7f3efc148e50 2024-04-02T16:11:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 -- 172.21.15.40:0/1493130856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc148e50 msgr2=0x7f3efc1492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc148e50 0x7f3efc1492c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c4d0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 -- 172.21.15.40:0/1493130856 shutdown_connections 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc148e50 0x7f3efc1492c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/1493130856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc0b23d0 0x7f3efc0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 -- 172.21.15.40:0/1493130856 >> 172.21.15.40:0/1493130856 conn(0x7f3efc0b8fd0 msgr2=0x7f3efc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 -- 172.21.15.40:0/1493130856 shutdown_connections 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 -- 172.21.15.40:0/1493130856 wait complete. 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.333+0000 7f3f03f3e640 1 Processor -- start 2024-04-02T16:11:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- start start 2024-04-02T16:11:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc0b23d0 0x7f3efc1e2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc148e50 0x7f3efc145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc152810 con 0x7f3efc073230 2024-04-02T16:11:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc152690 con 0x7f3efc0b23d0 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3efc152990 con 0x7f3efc148e50 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36274/0 (socket says 172.21.15.40:36274) 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 -- 172.21.15.40:0/4281210161 learned_addr learned my addr 172.21.15.40:0/4281210161 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 -- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc148e50 msgr2=0x7f3efc145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc148e50 0x7f3efc145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 -- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc0b23d0 msgr2=0x7f3efc1e2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc0b23d0 0x7f3efc1e2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 -- 172.21.15.40:0/4281210161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc145f90 con 0x7f3efc073230 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f01cda640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800d830 tx=0x7f3ee800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800b6c0 con 0x7f3efc073230 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8019070 con 0x7f3efc073230 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800ba80 con 0x7f3efc073230 2024-04-02T16:11:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1440c0 con 0x7f3efc073230 2024-04-02T16:11:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc13ad30 con 0x7f3efc073230 2024-04-02T16:11:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc0685f0 con 0x7f3efc073230 2024-04-02T16:11:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3ee801e460 con 0x7f3efc073230 2024-04-02T16:11:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 0x7f3ee006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ee8087030 con 0x7f3efc073230 2024-04-02T16:11:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.337+0000 7f3f014d9640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 0x7f3ee006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.341+0000 7f3f014d9640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 0x7f3ee006a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007fd0 tx=0x7f3ef0007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.341+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3efc145f90 con 0x7f3efc073230 2024-04-02T16:11:34.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:34.545+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3efc0b1690 con 0x7f3efc073230 2024-04-02T16:11:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.169+0000 7f3efa7fc640 1 -- 172.21.15.40:0/4281210161 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f3ee8057680 con 0x7f3efc073230 2024-04-02T16:11:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 msgr2=0x7f3ee006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 0x7f3ee006a870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007fd0 tx=0x7f3ef0007d60 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 msgr2=0x7f3efc13de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800d830 tx=0x7f3ee800dcf0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 shutdown_connections 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3efc148e50 0x7f3efc145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3ee00683b0 0x7f3ee006a870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc0b23d0 0x7f3efc1e2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 --2- 172.21.15.40:0/4281210161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3efc073230 0x7f3efc13de00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 >> 172.21.15.40:0/4281210161 conn(0x7f3efc0b8fd0 msgr2=0x7f3efc06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 shutdown_connections 2024-04-02T16:11:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.173+0000 7f3f03f3e640 1 -- 172.21.15.40:0/4281210161 wait complete. 2024-04-02T16:11:35.196 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-02T16:11:35.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.317+0000 7f134694b640 1 Processor -- start 2024-04-02T16:11:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.317+0000 7f134694b640 1 -- start start 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f134694b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f1340145550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f134694b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13400b3cc0 con 0x7f1340072ab0 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f134694b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1340145c10 con 0x7f1340072e90 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f134694b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1340148ac0 con 0x7f134006f280 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1340072ab0 0x7f134006e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50766/0 (socket says 172.21.15.40:50766) 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133ffff640 1 -- 172.21.15.40:0/443158650 learned_addr learned my addr 172.21.15.40:0/443158650 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f1340145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366327364 0 0) 0x7f13400b3cc0 con 0x7f1340072ab0 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f132c003530 con 0x7f1340072ab0 2024-04-02T16:11:35.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2501937873 0 0) 0x7f132c003530 con 0x7f1340072ab0 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 msgr2=0x7f1340145550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f1340145550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 >> v1:172.21.15.204:6789/0 conn(0x7f1340072e90 legacy=0x7f134014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1340148da0 con 0x7f1340072ab0 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f1330001680 con 0x7f1340072ab0 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 >> v1:172.21.15.40:6789/0 conn(0x7f1340072ab0 legacy=0x7f134006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132c0039f0 0x7f132c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f132c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f132c005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1340148ac0 con 0x7f132c004810 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f132c00bb60 con 0x7f132c0039f0 2024-04-02T16:11:35.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f132c00bd60 con 0x7f134006f280 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f132c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f133ffff640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132c0039f0 0x7f132c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f1344ee8640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f132c005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 -- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 msgr2=0x7f132c005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f134006f280 0x7f132c005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 -- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132c0039f0 msgr2=0x7f132c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f132c0039f0 0x7f132c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.321+0000 7f13377fe640 1 -- 172.21.15.40:0/443158650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c003530 con 0x7f132c004810 2024-04-02T16:11:35.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f13377fe640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f132c004c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1328009ae0 tx=0x7f132801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf40cc1611365d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b2070 con 0x7f132c004810 2024-04-02T16:11:35.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280a5530 con 0x7f132c004810 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133f7fe640 1 -- 172.21.15.40:0/443158650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280096b0 con 0x7f132c004810 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/443158650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 msgr2=0x7f132c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 --2- 172.21.15.40:0/443158650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f132c004c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1328009ae0 tx=0x7f132801ca70 comp rx=0 tx=0).stop 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f1344ee8640 1 -- 172.21.15.40:0/443158650 reap_dead start 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/443158650 shutdown_connections 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/443158650 >> 172.21.15.40:0/443158650 conn(0x7f13400b7f50 msgr2=0x7f13400b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/443158650 shutdown_connections 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/443158650 wait complete. 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 Processor -- start 2024-04-02T16:11:35.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- start start 2024-04-02T16:11:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1340141bc0 0x7f1340140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13401de290 0x7f13401de710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1340149a00 con 0x7f132c004810 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1340149880 con 0x7f1340141bc0 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f134006eef0 con 0x7f13401de290 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36984/0 (socket says 172.21.15.40:36984) 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 -- 172.21.15.40:0/2656577040 learned_addr learned my addr 172.21.15.40:0/2656577040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f13377fe640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1340141bc0 0x7f1340140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f1344ee8640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13401de290 0x7f13401de710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 -- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13401de290 msgr2=0x7f13401de710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13401de290 0x7f13401de710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 -- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1340141bc0 msgr2=0x7f1340140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1340141bc0 0x7f1340140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 -- 172.21.15.40:0/2656577040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13401420a0 con 0x7f132c004810 2024-04-02T16:11:35.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133ffff640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f133000e7f0 tx=0x7f133000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000c8d0 con 0x7f132c004810 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1330018070 con 0x7f132c004810 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134006e210 con 0x7f132c004810 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000ccd0 con 0x7f132c004810 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.325+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134006cfe0 con 0x7f132c004810 2024-04-02T16:11:35.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1340068d00 con 0x7f132c004810 2024-04-02T16:11:35.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f133001d460 con 0x7f132c004810 2024-04-02T16:11:35.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f133d7fa640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 0x7f132006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f13300867f0 con 0x7f132c004810 2024-04-02T16:11:35.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f13377fe640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 0x7f132006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.329+0000 7f13377fe640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 0x7f132006a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1328009ae0 tx=0x7f13280046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.333+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f1330056db0 con 0x7f132c004810 2024-04-02T16:11:35.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.529+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1340144820 con 0x7f132c004810 2024-04-02T16:11:35.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.529+0000 7f133d7fa640 1 -- 172.21.15.40:0/2656577040 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7f13300566b0 con 0x7f132c004810 2024-04-02T16:11:35.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:35.535 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:11:35.536 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:11:35.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 msgr2=0x7f132006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 0x7f132006a870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1328009ae0 tx=0x7f13280046d0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 msgr2=0x7f1340142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f133000e7f0 tx=0x7f133000ecb0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 shutdown_connections 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13401de290 0x7f13401de710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f13200683b0 0x7f132006a870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1340141bc0 0x7f1340140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 --2- 172.21.15.40:0/2656577040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f132c004810 0x7f1340142b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 >> 172.21.15.40:0/2656577040 conn(0x7f13400b7f50 msgr2=0x7f13400b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 shutdown_connections 2024-04-02T16:11:35.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.533+0000 7f134694b640 1 -- 172.21.15.40:0/2656577040 wait complete. 2024-04-02T16:11:35.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.669+0000 7f6b7fa62640 1 Processor -- start 2024-04-02T16:11:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 -- start start 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6b780b5c10 0x7f6b7802c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 0x7f6b780b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b780b5ff0 con 0x7f6b780b1770 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6b780b77e0 con 0x7f6b780b5c10 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7fa62640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7802cc30 con 0x7f6b7813fae0 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7cffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6b780b1770 0x7f6b7813f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50780/0 (socket says 172.21.15.40:50780) 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7cffd640 1 -- 172.21.15.40:0/1086710997 learned_addr learned my addr 172.21.15.40:0/1086710997 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7d7fe640 1 --2- 172.21.15.40:0/1086710997 >> v2:172.21.15.204:3300/0 conn(0x7f6b780b5c10 0x7f6b7802c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 0x7f6b780b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132378420 0 0) 0x7f6b780b5ff0 con 0x7f6b780b1770 2024-04-02T16:11:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b64003530 con 0x7f6b780b1770 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3518291542 0 0) 0x7f6b64003530 con 0x7f6b780b1770 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 msgr2=0x7f6b780b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 0x7f6b780b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 >> v2:172.21.15.204:3300/0 conn(0x7f6b780b5c10 msgr2=0x7f6b7802c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 --2- 172.21.15.40:0/1086710997 >> v2:172.21.15.204:3300/0 conn(0x7f6b780b5c10 0x7f6b7802c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b780b2ff0 con 0x7f6b780b1770 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f6b68001800 con 0x7f6b780b1770 2024-04-02T16:11:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 >> v1:172.21.15.40:6789/0 conn(0x7f6b780b1770 legacy=0x7f6b7813f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 0x7f6b64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b780b77e0 con 0x7f6b64004d50 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7802cc30 con 0x7f6b640039c0 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6400ba50 con 0x7f6b7813fae0 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 -- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 msgr2=0x7f6b64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b7813fae0 0x7f6b64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 -- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b640039c0 msgr2=0x7f6b64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b640039c0 0x7f6b64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.673+0000 7f6b7dfff640 1 -- 172.21.15.40:0/1086710997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b64003530 con 0x7f6b64004d50 2024-04-02T16:11:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009fc0 tx=0x7f6b7001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5c627f0f1dd034 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b700b2070 con 0x7f6b64004d50 2024-04-02T16:11:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b700a5ab0 con 0x7f6b64004d50 2024-04-02T16:11:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b6ffff640 1 -- 172.21.15.40:0/1086710997 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70009c10 con 0x7f6b64004d50 2024-04-02T16:11:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- 172.21.15.40:0/1086710997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b64004d50 msgr2=0x7f6b640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/1086710997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b64004d50 0x7f6b640051d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009fc0 tx=0x7f6b7001ccc0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7dfff640 1 -- 172.21.15.40:0/1086710997 reap_dead start 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- 172.21.15.40:0/1086710997 shutdown_connections 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- 172.21.15.40:0/1086710997 >> 172.21.15.40:0/1086710997 conn(0x7f6b780bb3e0 msgr2=0x7f6b780bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- 172.21.15.40:0/1086710997 shutdown_connections 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- 172.21.15.40:0/1086710997 wait complete. 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 Processor -- start 2024-04-02T16:11:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- start start 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b64004d50 0x7f6b781404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7814da40 0x7f6b78136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b781508a0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b78150720 con 0x7f6b7814da40 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7fa62640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6b78150a20 con 0x7f6b64004d50 2024-04-02T16:11:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36994/0 (socket says 172.21.15.40:36994) 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 -- 172.21.15.40:0/2573118348 learned_addr learned my addr 172.21.15.40:0/2573118348 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7dfff640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7814da40 0x7f6b78136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 -- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b64004d50 msgr2=0x7f6b781404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b64004d50 0x7f6b781404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 -- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7814da40 msgr2=0x7f6b78136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7814da40 0x7f6b78136cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.677+0000 7f6b7cffd640 1 -- 172.21.15.40:0/2573118348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b78142040 con 0x7f6b7814d5c0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7cffd640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c680 tx=0x7f6b6800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68017070 con 0x7f6b7814d5c0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68007290 con 0x7f6b7814d5c0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b78142320 con 0x7f6b7814d5c0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680057b0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b781418d0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b78068d00 con 0x7f6b7814d5c0 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6b6801a070 con 0x7f6b7814d5c0 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 0x7f6b5c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b68085da0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7d7fe640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 0x7f6b5c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.681+0000 7f6b7d7fe640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 0x7f6b5c06a840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006aa0 tx=0x7f6b74004780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.685+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f6b680563f0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.849+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b78072f10 con 0x7f6b7814d5c0 2024-04-02T16:11:35.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b6dffb640 1 -- 172.21.15.40:0/2573118348 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f6b68055cf0 con 0x7f6b7814d5c0 2024-04-02T16:11:35.856 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:35.857 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 msgr2=0x7f6b5c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 0x7f6b5c06a840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006aa0 tx=0x7f6b74004780 comp rx=0 tx=0).stop 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 msgr2=0x7f6b781e3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800c680 tx=0x7f6b6800cb40 comp rx=0 tx=0).stop 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 shutdown_connections 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b5c068380 0x7f6b5c06a840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b7814da40 0x7f6b78136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b7814d5c0 0x7f6b781e3bb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 --2- 172.21.15.40:0/2573118348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b64004d50 0x7f6b781404c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 >> 172.21.15.40:0/2573118348 conn(0x7f6b780bb3e0 msgr2=0x7f6b780b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 shutdown_connections 2024-04-02T16:11:35.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.853+0000 7f6b7fa62640 1 -- 172.21.15.40:0/2573118348 wait complete. 2024-04-02T16:11:35.876 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T16:11:35.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:11:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 Processor -- start 2024-04-02T16:11:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 -- start start 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd86c0b4bf0 0x7fd86c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86c0b6310 con 0x7fd86c0b3500 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd86c0b55d0 con 0x7fd86c0b8c80 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd87411c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd86c14b560 con 0x7fd86c0b4bf0 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd871eb8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd86c0b4bf0 0x7fd86c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd8716b7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd86c0b3500 0x7fd86c14ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50796/0 (socket says 172.21.15.40:50796) 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd8716b7640 1 -- 172.21.15.40:0/3847972291 learned_addr learned my addr 172.21.15.40:0/3847972291 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd871eb8640 1 -- 172.21.15.40:0/3847972291 >> v1:172.21.15.204:6789/0 conn(0x7fd86c0b8c80 legacy=0x7fd86c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd871eb8640 1 -- 172.21.15.40:0/3847972291 >> v1:172.21.15.40:6789/0 conn(0x7fd86c0b3500 legacy=0x7fd86c14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd871eb8640 1 -- 172.21.15.40:0/3847972291 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c0b3fa0 con 0x7fd86c0b4bf0 2024-04-02T16:11:35.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd871eb8640 1 --2- 172.21.15.40:0/3847972291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c0b4fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097c0 tx=0x7fd86801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4911a2cf27e0a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:35.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd870eb6640 1 -- 172.21.15.40:0/3847972291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680b0070 con 0x7fd86c0b4bf0 2024-04-02T16:11:35.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd870eb6640 1 -- 172.21.15.40:0/3847972291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8680ab600 con 0x7fd86c0b4bf0 2024-04-02T16:11:35.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.989+0000 7fd870eb6640 1 -- 172.21.15.40:0/3847972291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680ab9d0 con 0x7fd86c0b4bf0 2024-04-02T16:11:35.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/3847972291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 msgr2=0x7fd86c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:35.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 --2- 172.21.15.40:0/3847972291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c0b4fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097c0 tx=0x7fd86801c340 comp rx=0 tx=0).stop 2024-04-02T16:11:35.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/3847972291 shutdown_connections 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 --2- 172.21.15.40:0/3847972291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c0b4fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/3847972291 >> 172.21.15.40:0/3847972291 conn(0x7fd86c0734b0 msgr2=0x7fd86c0bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/3847972291 shutdown_connections 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/3847972291 wait complete. 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 Processor -- start 2024-04-02T16:11:35.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- start start 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd86c0b3500 0x7fd86c1e4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 0x7fd86c13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c156740 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1568c0 con 0x7fd86c0b3500 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c156a40 con 0x7fd86c0b8c80 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37008/0 (socket says 172.21.15.40:37008) 2024-04-02T16:11:36.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 -- 172.21.15.40:0/868732997 learned_addr learned my addr 172.21.15.40:0/868732997 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8726b9640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 0x7fd86c13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 -- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 msgr2=0x7fd86c13ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 0x7fd86c13ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 -- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd86c0b3500 msgr2=0x7fd86c1e4730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd86c0b3500 0x7fd86c1e4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 -- 172.21.15.40:0/868732997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c145830 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8726b9640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 0x7fd86c13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd8716b7640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c420 tx=0x7fd86000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860017070 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd860005cf0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8600072f0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c1e4ed0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.993+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c13c7c0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c0685f0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd86001a070 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd85e7fc640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 0x7fd85006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd860085ca0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd871eb8640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 0x7fd85006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:35.997+0000 7fd871eb8640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 0x7fd85006a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ad000 tx=0x7fd868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.001+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd8600562f0 con 0x7fd86c0b4bf0 2024-04-02T16:11:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.153+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd86c0bcd50 con 0x7fd850068380 2024-04-02T16:11:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.173+0000 7fd85e7fc640 1 -- 172.21.15.40:0/868732997 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251919 (secure 0 0 0) 0x7fd86c0bcd50 con 0x7fd850068380 2024-04-02T16:11:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:36.184 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 msgr2=0x7fd85006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 0x7fd85006a840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ad000 tx=0x7fd868008040 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 msgr2=0x7fd86c13c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c420 tx=0x7fd86000c8e0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 shutdown_connections 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd86c0b8c80 0x7fd86c13ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd850068380 0x7fd85006a840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd86c0b4bf0 0x7fd86c13c280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 --2- 172.21.15.40:0/868732997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd86c0b3500 0x7fd86c1e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 >> 172.21.15.40:0/868732997 conn(0x7fd86c0734b0 msgr2=0x7fd86c1355f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:36.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 shutdown_connections 2024-04-02T16:11:36.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.181+0000 7fd87411c640 1 -- 172.21.15.40:0/868732997 wait complete. 2024-04-02T16:11:36.199 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-02T16:11:35.848781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218972,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755748,"statfs":{"total":773094113280,"available":772869885952,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753044"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497949+0000","last_change":"2024-04-02T16:11:32.176361+0000","last_active":"2024-04-02T16:11:33.497949+0000","last_peered":"2024-04-02T16:11:33.497949+0000","last_clean":"2024-04-02T16:11:33.497949+0000","last_became_active":"2024-04-02T16:11:32.176117+0000","last_became_peered":"2024-04-02T16:11:32.176117+0000","last_unstale":"2024-04-02T16:11:33.497949+0000","last_undegraded":"2024-04-02T16:11:33.497949+0000","last_fullsized":"2024-04-02T16:11:33.497949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:03.664122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156872+0000","last_change":"2024-04-02T16:11:32.165208+0000","last_active":"2024-04-02T16:11:33.156872+0000","last_peered":"2024-04-02T16:11:33.156872+0000","last_clean":"2024-04-02T16:11:33.156872+0000","last_became_active":"2024-04-02T16:11:32.164869+0000","last_became_peered":"2024-04-02T16:11:32.164869+0000","last_unstale":"2024-04-02T16:11:33.156872+0000","last_undegraded":"2024-04-02T16:11:33.156872+0000","last_fullsized":"2024-04-02T16:11:33.156872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158555+0000","last_change":"2024-04-02T16:11:32.146435+0000","last_active":"2024-04-02T16:11:32.158555+0000","last_peered":"2024-04-02T16:11:32.158555+0000","last_clean":"2024-04-02T16:11:32.158555+0000","last_became_active":"2024-04-02T16:11:32.145704+0000","last_became_peered":"2024-04-02T16:11:32.145704+0000","last_unstale":"2024-04-02T16:11:32.158555+0000","last_undegraded":"2024-04-02T16:11:32.158555+0000","last_fullsized":"2024-04-02T16:11:32.158555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:44.133779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158980+0000","last_change":"2024-04-02T16:11:32.172527+0000","last_active":"2024-04-02T16:11:33.158980+0000","last_peered":"2024-04-02T16:11:33.158980+0000","last_clean":"2024-04-02T16:11:33.158980+0000","last_became_active":"2024-04-02T16:11:32.170021+0000","last_became_peered":"2024-04-02T16:11:32.170021+0000","last_unstale":"2024-04-02T16:11:33.158980+0000","last_undegraded":"2024-04-02T16:11:33.158980+0000","last_fullsized":"2024-04-02T16:11:33.158980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497274+0000","last_change":"2024-04-02T16:11:32.171344+0000","last_active":"2024-04-02T16:11:33.497274+0000","last_peered":"2024-04-02T16:11:33.497274+0000","last_clean":"2024-04-02T16:11:33.497274+0000","last_became_active":"2024-04-02T16:11:32.170838+0000","last_became_peered":"2024-04-02T16:11:32.170838+0000","last_unstale":"2024-04-02T16:11:33.497274+0000","last_undegraded":"2024-04-02T16:11:33.497274+0000","last_fullsized":"2024-04-02T16:11:33.497274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158169+0000","last_change":"2024-04-02T16:11:32.160886+0000","last_active":"2024-04-02T16:11:33.158169+0000","last_peered":"2024-04-02T16:11:33.158169+0000","last_clean":"2024-04-02T16:11:33.158169+0000","last_became_active":"2024-04-02T16:11:32.159998+0000","last_became_peered":"2024-04-02T16:11:32.159998+0000","last_unstale":"2024-04-02T16:11:33.158169+0000","last_undegraded":"2024-04-02T16:11:33.158169+0000","last_fullsized":"2024-04-02T16:11:33.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496848+0000","last_change":"2024-04-02T16:11:32.172231+0000","last_active":"2024-04-02T16:11:33.496848+0000","last_peered":"2024-04-02T16:11:33.496848+0000","last_clean":"2024-04-02T16:11:33.496848+0000","last_became_active":"2024-04-02T16:11:32.171852+0000","last_became_peered":"2024-04-02T16:11:32.171852+0000","last_unstale":"2024-04-02T16:11:33.496848+0000","last_undegraded":"2024-04-02T16:11:33.496848+0000","last_fullsized":"2024-04-02T16:11:33.496848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178509+0000","last_change":"2024-04-02T16:11:32.171342+0000","last_active":"2024-04-02T16:11:32.178509+0000","last_peered":"2024-04-02T16:11:32.178509+0000","last_clean":"2024-04-02T16:11:32.178509+0000","last_became_active":"2024-04-02T16:11:32.171027+0000","last_became_peered":"2024-04-02T16:11:32.171027+0000","last_unstale":"2024-04-02T16:11:32.178509+0000","last_undegraded":"2024-04-02T16:11:32.178509+0000","last_fullsized":"2024-04-02T16:11:32.178509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:55.127710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498216+0000","last_change":"2024-04-02T16:11:32.171461+0000","last_active":"2024-04-02T16:11:33.498216+0000","last_peered":"2024-04-02T16:11:33.498216+0000","last_clean":"2024-04-02T16:11:33.498216+0000","last_became_active":"2024-04-02T16:11:32.170971+0000","last_became_peered":"2024-04-02T16:11:32.170971+0000","last_unstale":"2024-04-02T16:11:33.498216+0000","last_undegraded":"2024-04-02T16:11:33.498216+0000","last_fullsized":"2024-04-02T16:11:33.498216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:25.000722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161307+0000","last_change":"2024-04-02T16:11:32.164006+0000","last_active":"2024-04-02T16:11:34.161307+0000","last_peered":"2024-04-02T16:11:34.161307+0000","last_clean":"2024-04-02T16:11:34.161307+0000","last_became_active":"2024-04-02T16:11:32.163712+0000","last_became_peered":"2024-04-02T16:11:32.163712+0000","last_unstale":"2024-04-02T16:11:34.161307+0000","last_undegraded":"2024-04-02T16:11:34.161307+0000","last_fullsized":"2024-04-02T16:11:34.161307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160963+0000","last_change":"2024-04-02T16:11:32.162643+0000","last_active":"2024-04-02T16:11:34.160963+0000","last_peered":"2024-04-02T16:11:34.160963+0000","last_clean":"2024-04-02T16:11:34.160963+0000","last_became_active":"2024-04-02T16:11:32.162329+0000","last_became_peered":"2024-04-02T16:11:32.162329+0000","last_unstale":"2024-04-02T16:11:34.160963+0000","last_undegraded":"2024-04-02T16:11:34.160963+0000","last_fullsized":"2024-04-02T16:11:34.160963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158378+0000","last_change":"2024-04-02T16:11:32.150507+0000","last_active":"2024-04-02T16:11:33.158378+0000","last_peered":"2024-04-02T16:11:33.158378+0000","last_clean":"2024-04-02T16:11:33.158378+0000","last_became_active":"2024-04-02T16:11:32.149817+0000","last_became_peered":"2024-04-02T16:11:32.149817+0000","last_unstale":"2024-04-02T16:11:33.158378+0000","last_undegraded":"2024-04-02T16:11:33.158378+0000","last_fullsized":"2024-04-02T16:11:33.158378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178417+0000","last_change":"2024-04-02T16:11:32.171214+0000","last_active":"2024-04-02T16:11:32.178417+0000","last_peered":"2024-04-02T16:11:32.178417+0000","last_clean":"2024-04-02T16:11:32.178417+0000","last_became_active":"2024-04-02T16:11:32.170528+0000","last_became_peered":"2024-04-02T16:11:32.170528+0000","last_unstale":"2024-04-02T16:11:32.178417+0000","last_undegraded":"2024-04-02T16:11:32.178417+0000","last_fullsized":"2024-04-02T16:11:32.178417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158518+0000","last_change":"2024-04-02T16:11:32.148238+0000","last_active":"2024-04-02T16:11:32.158518+0000","last_peered":"2024-04-02T16:11:32.158518+0000","last_clean":"2024-04-02T16:11:32.158518+0000","last_became_active":"2024-04-02T16:11:32.147348+0000","last_became_peered":"2024-04-02T16:11:32.147348+0000","last_unstale":"2024-04-02T16:11:32.158518+0000","last_undegraded":"2024-04-02T16:11:32.158518+0000","last_fullsized":"2024-04-02T16:11:32.158518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:23.736628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156485+0000","last_change":"2024-04-02T16:11:32.174912+0000","last_active":"2024-04-02T16:11:33.156485+0000","last_peered":"2024-04-02T16:11:33.156485+0000","last_clean":"2024-04-02T16:11:33.156485+0000","last_became_active":"2024-04-02T16:11:32.174581+0000","last_became_peered":"2024-04-02T16:11:32.174581+0000","last_unstale":"2024-04-02T16:11:33.156485+0000","last_undegraded":"2024-04-02T16:11:33.156485+0000","last_fullsized":"2024-04-02T16:11:33.156485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156962+0000","last_change":"2024-04-02T16:11:32.160602+0000","last_active":"2024-04-02T16:11:33.156962+0000","last_peered":"2024-04-02T16:11:33.156962+0000","last_clean":"2024-04-02T16:11:33.156962+0000","last_became_active":"2024-04-02T16:11:32.160289+0000","last_became_peered":"2024-04-02T16:11:32.160289+0000","last_unstale":"2024-04-02T16:11:33.156962+0000","last_undegraded":"2024-04-02T16:11:33.156962+0000","last_fullsized":"2024-04-02T16:11:33.156962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159486+0000","last_change":"2024-04-02T16:11:32.159580+0000","last_active":"2024-04-02T16:11:33.159486+0000","last_peered":"2024-04-02T16:11:33.159486+0000","last_clean":"2024-04-02T16:11:33.159486+0000","last_became_active":"2024-04-02T16:11:32.159248+0000","last_became_peered":"2024-04-02T16:11:32.159248+0000","last_unstale":"2024-04-02T16:11:33.159486+0000","last_undegraded":"2024-04-02T16:11:33.159486+0000","last_fullsized":"2024-04-02T16:11:33.159486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.156055+0000","last_change":"2024-04-02T16:11:32.152173+0000","last_active":"2024-04-02T16:11:32.156055+0000","last_peered":"2024-04-02T16:11:32.156055+0000","last_clean":"2024-04-02T16:11:32.156055+0000","last_became_active":"2024-04-02T16:11:32.150901+0000","last_became_peered":"2024-04-02T16:11:32.150901+0000","last_unstale":"2024-04-02T16:11:32.156055+0000","last_undegraded":"2024-04-02T16:11:32.156055+0000","last_fullsized":"2024-04-02T16:11:32.156055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497849+0000","last_change":"2024-04-02T16:11:32.174590+0000","last_active":"2024-04-02T16:11:33.497849+0000","last_peered":"2024-04-02T16:11:33.497849+0000","last_clean":"2024-04-02T16:11:33.497849+0000","last_became_active":"2024-04-02T16:11:32.174269+0000","last_became_peered":"2024-04-02T16:11:32.174269+0000","last_unstale":"2024-04-02T16:11:33.497849+0000","last_undegraded":"2024-04-02T16:11:33.497849+0000","last_fullsized":"2024-04-02T16:11:33.497849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160975+0000","last_change":"2024-04-02T16:11:32.174103+0000","last_active":"2024-04-02T16:11:34.160975+0000","last_peered":"2024-04-02T16:11:34.160975+0000","last_clean":"2024-04-02T16:11:34.160975+0000","last_became_active":"2024-04-02T16:11:32.173844+0000","last_became_peered":"2024-04-02T16:11:32.173844+0000","last_unstale":"2024-04-02T16:11:34.160975+0000","last_undegraded":"2024-04-02T16:11:34.160975+0000","last_fullsized":"2024-04-02T16:11:34.160975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.170281+0000","last_change":"2024-04-02T16:11:32.155248+0000","last_active":"2024-04-02T16:11:32.170281+0000","last_peered":"2024-04-02T16:11:32.170281+0000","last_clean":"2024-04-02T16:11:32.170281+0000","last_became_active":"2024-04-02T16:11:32.154493+0000","last_became_peered":"2024-04-02T16:11:32.154493+0000","last_unstale":"2024-04-02T16:11:32.170281+0000","last_undegraded":"2024-04-02T16:11:32.170281+0000","last_fullsized":"2024-04-02T16:11:32.170281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158188+0000","last_change":"2024-04-02T16:11:32.163843+0000","last_active":"2024-04-02T16:11:33.158188+0000","last_peered":"2024-04-02T16:11:33.158188+0000","last_clean":"2024-04-02T16:11:33.158188+0000","last_became_active":"2024-04-02T16:11:32.163493+0000","last_became_peered":"2024-04-02T16:11:32.163493+0000","last_unstale":"2024-04-02T16:11:33.158188+0000","last_undegraded":"2024-04-02T16:11:33.158188+0000","last_fullsized":"2024-04-02T16:11:33.158188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.166023+0000","last_change":"2024-04-02T16:11:32.160644+0000","last_active":"2024-04-02T16:11:32.166023+0000","last_peered":"2024-04-02T16:11:32.166023+0000","last_clean":"2024-04-02T16:11:32.166023+0000","last_became_active":"2024-04-02T16:11:32.160265+0000","last_became_peered":"2024-04-02T16:11:32.160265+0000","last_unstale":"2024-04-02T16:11:32.166023+0000","last_undegraded":"2024-04-02T16:11:32.166023+0000","last_fullsized":"2024-04-02T16:11:32.166023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:44.133779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157349+0000","last_change":"2024-04-02T16:11:32.169639+0000","last_active":"2024-04-02T16:11:33.157349+0000","last_peered":"2024-04-02T16:11:33.157349+0000","last_clean":"2024-04-02T16:11:33.157349+0000","last_became_active":"2024-04-02T16:11:32.166290+0000","last_became_peered":"2024-04-02T16:11:32.166290+0000","last_unstale":"2024-04-02T16:11:33.157349+0000","last_undegraded":"2024-04-02T16:11:33.157349+0000","last_fullsized":"2024-04-02T16:11:33.157349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179020+0000","last_change":"2024-04-02T16:11:32.172667+0000","last_active":"2024-04-02T16:11:32.179020+0000","last_peered":"2024-04-02T16:11:32.179020+0000","last_clean":"2024-04-02T16:11:32.179020+0000","last_became_active":"2024-04-02T16:11:32.171185+0000","last_became_peered":"2024-04-02T16:11:32.171185+0000","last_unstale":"2024-04-02T16:11:32.179020+0000","last_undegraded":"2024-04-02T16:11:32.179020+0000","last_fullsized":"2024-04-02T16:11:32.179020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:36.835221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161390+0000","last_change":"2024-04-02T16:11:32.174173+0000","last_active":"2024-04-02T16:11:34.161390+0000","last_peered":"2024-04-02T16:11:34.161390+0000","last_clean":"2024-04-02T16:11:34.161390+0000","last_became_active":"2024-04-02T16:11:32.173799+0000","last_became_peered":"2024-04-02T16:11:32.173799+0000","last_unstale":"2024-04-02T16:11:34.161390+0000","last_undegraded":"2024-04-02T16:11:34.161390+0000","last_fullsized":"2024-04-02T16:11:34.161390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156692+0000","last_change":"2024-04-02T16:11:32.169323+0000","last_active":"2024-04-02T16:11:33.156692+0000","last_peered":"2024-04-02T16:11:33.156692+0000","last_clean":"2024-04-02T16:11:33.156692+0000","last_became_active":"2024-04-02T16:11:32.169028+0000","last_became_peered":"2024-04-02T16:11:32.169028+0000","last_unstale":"2024-04-02T16:11:33.156692+0000","last_undegraded":"2024-04-02T16:11:33.156692+0000","last_fullsized":"2024-04-02T16:11:33.156692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179222+0000","last_change":"2024-04-02T16:11:32.177772+0000","last_active":"2024-04-02T16:11:32.179222+0000","last_peered":"2024-04-02T16:11:32.179222+0000","last_clean":"2024-04-02T16:11:32.179222+0000","last_became_active":"2024-04-02T16:11:32.177123+0000","last_became_peered":"2024-04-02T16:11:32.177123+0000","last_unstale":"2024-04-02T16:11:32.179222+0000","last_undegraded":"2024-04-02T16:11:32.179222+0000","last_fullsized":"2024-04-02T16:11:32.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:28:10.993274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.171094+0000","last_change":"2024-04-02T16:11:32.168117+0000","last_active":"2024-04-02T16:11:32.171094+0000","last_peered":"2024-04-02T16:11:32.171094+0000","last_clean":"2024-04-02T16:11:32.171094+0000","last_became_active":"2024-04-02T16:11:32.167687+0000","last_became_peered":"2024-04-02T16:11:32.167687+0000","last_unstale":"2024-04-02T16:11:32.171094+0000","last_undegraded":"2024-04-02T16:11:32.171094+0000","last_fullsized":"2024-04-02T16:11:32.171094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:36.835221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158588+0000","last_change":"2024-04-02T16:11:32.168501+0000","last_active":"2024-04-02T16:11:33.158588+0000","last_peered":"2024-04-02T16:11:33.158588+0000","last_clean":"2024-04-02T16:11:33.158588+0000","last_became_active":"2024-04-02T16:11:32.168164+0000","last_became_peered":"2024-04-02T16:11:32.168164+0000","last_unstale":"2024-04-02T16:11:33.158588+0000","last_undegraded":"2024-04-02T16:11:33.158588+0000","last_fullsized":"2024-04-02T16:11:33.158588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.157254+0000","last_change":"2024-04-02T16:11:32.152124+0000","last_active":"2024-04-02T16:11:32.157254+0000","last_peered":"2024-04-02T16:11:32.157254+0000","last_clean":"2024-04-02T16:11:32.157254+0000","last_became_active":"2024-04-02T16:11:32.150859+0000","last_became_peered":"2024-04-02T16:11:32.150859+0000","last_unstale":"2024-04-02T16:11:32.157254+0000","last_undegraded":"2024-04-02T16:11:32.157254+0000","last_fullsized":"2024-04-02T16:11:32.157254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160495+0000","last_change":"2024-04-02T16:11:32.166672+0000","last_active":"2024-04-02T16:11:34.160495+0000","last_peered":"2024-04-02T16:11:34.160495+0000","last_clean":"2024-04-02T16:11:34.160495+0000","last_became_active":"2024-04-02T16:11:32.165841+0000","last_became_peered":"2024-04-02T16:11:32.165841+0000","last_unstale":"2024-04-02T16:11:34.160495+0000","last_undegraded":"2024-04-02T16:11:34.160495+0000","last_fullsized":"2024-04-02T16:11:34.160495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157143+0000","last_change":"2024-04-02T16:11:32.148450+0000","last_active":"2024-04-02T16:11:33.157143+0000","last_peered":"2024-04-02T16:11:33.157143+0000","last_clean":"2024-04-02T16:11:33.157143+0000","last_became_active":"2024-04-02T16:11:32.147722+0000","last_became_peered":"2024-04-02T16:11:32.147722+0000","last_unstale":"2024-04-02T16:11:33.157143+0000","last_undegraded":"2024-04-02T16:11:33.157143+0000","last_fullsized":"2024-04-02T16:11:33.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.172261+0000","last_change":"2024-04-02T16:11:31.123742+0000","last_active":"2024-04-02T16:11:32.172261+0000","last_peered":"2024-04-02T16:11:32.172261+0000","last_clean":"2024-04-02T16:11:32.172261+0000","last_became_active":"2024-04-02T16:11:31.123404+0000","last_became_peered":"2024-04-02T16:11:31.123404+0000","last_unstale":"2024-04-02T16:11:32.172261+0000","last_undegraded":"2024-04-02T16:11:32.172261+0000","last_fullsized":"2024-04-02T16:11:32.172261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158329+0000","last_change":"2024-04-02T16:11:32.140773+0000","last_active":"2024-04-02T16:11:32.158329+0000","last_peered":"2024-04-02T16:11:32.158329+0000","last_clean":"2024-04-02T16:11:32.158329+0000","last_became_active":"2024-04-02T16:11:32.140008+0000","last_became_peered":"2024-04-02T16:11:32.140008+0000","last_unstale":"2024-04-02T16:11:32.158329+0000","last_undegraded":"2024-04-02T16:11:32.158329+0000","last_fullsized":"2024-04-02T16:11:32.158329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:13.477320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160916+0000","last_change":"2024-04-02T16:11:31.131833+0000","last_active":"2024-04-02T16:11:34.160916+0000","last_peered":"2024-04-02T16:11:34.160916+0000","last_clean":"2024-04-02T16:11:34.160916+0000","last_became_active":"2024-04-02T16:11:31.130705+0000","last_became_peered":"2024-04-02T16:11:31.130705+0000","last_unstale":"2024-04-02T16:11:34.160916+0000","last_undegraded":"2024-04-02T16:11:34.160916+0000","last_fullsized":"2024-04-02T16:11:34.160916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.150813+0000","last_change":"2024-04-02T16:11:31.130860+0000","last_active":"2024-04-02T16:11:32.150813+0000","last_peered":"2024-04-02T16:11:32.150813+0000","last_clean":"2024-04-02T16:11:32.150813+0000","last_became_active":"2024-04-02T16:11:31.130391+0000","last_became_peered":"2024-04-02T16:11:31.130391+0000","last_unstale":"2024-04-02T16:11:32.150813+0000","last_undegraded":"2024-04-02T16:11:32.150813+0000","last_fullsized":"2024-04-02T16:11:32.150813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178184+0000","last_change":"2024-04-02T16:11:32.166977+0000","last_active":"2024-04-02T16:11:32.178184+0000","last_peered":"2024-04-02T16:11:32.178184+0000","last_clean":"2024-04-02T16:11:32.178184+0000","last_became_active":"2024-04-02T16:11:32.166690+0000","last_became_peered":"2024-04-02T16:11:32.166690+0000","last_unstale":"2024-04-02T16:11:32.178184+0000","last_undegraded":"2024-04-02T16:11:32.178184+0000","last_fullsized":"2024-04-02T16:11:32.178184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157628+0000","last_change":"2024-04-02T16:11:31.128418+0000","last_active":"2024-04-02T16:11:33.157628+0000","last_peered":"2024-04-02T16:11:33.157628+0000","last_clean":"2024-04-02T16:11:33.157628+0000","last_became_active":"2024-04-02T16:11:31.126329+0000","last_became_peered":"2024-04-02T16:11:31.126329+0000","last_unstale":"2024-04-02T16:11:33.157628+0000","last_undegraded":"2024-04-02T16:11:33.157628+0000","last_fullsized":"2024-04-02T16:11:33.157628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156477+0000","last_change":"2024-04-02T16:11:32.164197+0000","last_active":"2024-04-02T16:11:33.156477+0000","last_peered":"2024-04-02T16:11:33.156477+0000","last_clean":"2024-04-02T16:11:33.156477+0000","last_became_active":"2024-04-02T16:11:32.163440+0000","last_became_peered":"2024-04-02T16:11:32.163440+0000","last_unstale":"2024-04-02T16:11:33.156477+0000","last_undegraded":"2024-04-02T16:11:33.156477+0000","last_fullsized":"2024-04-02T16:11:33.156477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161113+0000","last_change":"2024-04-02T16:11:31.137185+0000","last_active":"2024-04-02T16:11:34.161113+0000","last_peered":"2024-04-02T16:11:34.161113+0000","last_clean":"2024-04-02T16:11:34.161113+0000","last_became_active":"2024-04-02T16:11:31.136386+0000","last_became_peered":"2024-04-02T16:11:31.136386+0000","last_unstale":"2024-04-02T16:11:34.161113+0000","last_undegraded":"2024-04-02T16:11:34.161113+0000","last_fullsized":"2024-04-02T16:11:34.161113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158908+0000","last_change":"2024-04-02T16:11:32.159269+0000","last_active":"2024-04-02T16:11:33.158908+0000","last_peered":"2024-04-02T16:11:33.158908+0000","last_clean":"2024-04-02T16:11:33.158908+0000","last_became_active":"2024-04-02T16:11:32.158906+0000","last_became_peered":"2024-04-02T16:11:32.158906+0000","last_unstale":"2024-04-02T16:11:33.158908+0000","last_undegraded":"2024-04-02T16:11:33.158908+0000","last_fullsized":"2024-04-02T16:11:33.158908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159157+0000","last_change":"2024-04-02T16:11:31.133206+0000","last_active":"2024-04-02T16:11:33.159157+0000","last_peered":"2024-04-02T16:11:33.159157+0000","last_clean":"2024-04-02T16:11:33.159157+0000","last_became_active":"2024-04-02T16:11:31.132012+0000","last_became_peered":"2024-04-02T16:11:31.132012+0000","last_unstale":"2024-04-02T16:11:33.159157+0000","last_undegraded":"2024-04-02T16:11:33.159157+0000","last_fullsized":"2024-04-02T16:11:33.159157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158578+0000","last_change":"2024-04-02T16:11:32.153558+0000","last_active":"2024-04-02T16:11:32.158578+0000","last_peered":"2024-04-02T16:11:32.158578+0000","last_clean":"2024-04-02T16:11:32.158578+0000","last_became_active":"2024-04-02T16:11:32.152697+0000","last_became_peered":"2024-04-02T16:11:32.152697+0000","last_unstale":"2024-04-02T16:11:32.158578+0000","last_undegraded":"2024-04-02T16:11:32.158578+0000","last_fullsized":"2024-04-02T16:11:32.158578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156779+0000","last_change":"2024-04-02T16:11:31.134656+0000","last_active":"2024-04-02T16:11:33.156779+0000","last_peered":"2024-04-02T16:11:33.156779+0000","last_clean":"2024-04-02T16:11:33.156779+0000","last_became_active":"2024-04-02T16:11:31.134191+0000","last_became_peered":"2024-04-02T16:11:31.134191+0000","last_unstale":"2024-04-02T16:11:33.156779+0000","last_undegraded":"2024-04-02T16:11:33.156779+0000","last_fullsized":"2024-04-02T16:11:33.156779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158486+0000","last_change":"2024-04-02T16:11:32.154312+0000","last_active":"2024-04-02T16:11:33.158486+0000","last_peered":"2024-04-02T16:11:33.158486+0000","last_clean":"2024-04-02T16:11:33.158486+0000","last_became_active":"2024-04-02T16:11:32.153984+0000","last_became_peered":"2024-04-02T16:11:32.153984+0000","last_unstale":"2024-04-02T16:11:33.158486+0000","last_undegraded":"2024-04-02T16:11:33.158486+0000","last_fullsized":"2024-04-02T16:11:33.158486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161022+0000","last_change":"2024-04-02T16:11:31.128529+0000","last_active":"2024-04-02T16:11:34.161022+0000","last_peered":"2024-04-02T16:11:34.161022+0000","last_clean":"2024-04-02T16:11:34.161022+0000","last_became_active":"2024-04-02T16:11:31.127760+0000","last_became_peered":"2024-04-02T16:11:31.127760+0000","last_unstale":"2024-04-02T16:11:34.161022+0000","last_undegraded":"2024-04-02T16:11:34.161022+0000","last_fullsized":"2024-04-02T16:11:34.161022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497089+0000","last_change":"2024-04-02T16:11:32.169840+0000","last_active":"2024-04-02T16:11:33.497089+0000","last_peered":"2024-04-02T16:11:33.497089+0000","last_clean":"2024-04-02T16:11:33.497089+0000","last_became_active":"2024-04-02T16:11:32.167352+0000","last_became_peered":"2024-04-02T16:11:32.167352+0000","last_unstale":"2024-04-02T16:11:33.497089+0000","last_undegraded":"2024-04-02T16:11:33.497089+0000","last_fullsized":"2024-04-02T16:11:33.497089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.174627+0000","last_change":"2024-04-02T16:11:31.135077+0000","last_active":"2024-04-02T16:11:32.174627+0000","last_peered":"2024-04-02T16:11:32.174627+0000","last_clean":"2024-04-02T16:11:32.174627+0000","last_became_active":"2024-04-02T16:11:31.134519+0000","last_became_peered":"2024-04-02T16:11:31.134519+0000","last_unstale":"2024-04-02T16:11:32.174627+0000","last_undegraded":"2024-04-02T16:11:32.174627+0000","last_fullsized":"2024-04-02T16:11:32.174627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.169908+0000","last_change":"2024-04-02T16:11:32.155398+0000","last_active":"2024-04-02T16:11:32.169908+0000","last_peered":"2024-04-02T16:11:32.169908+0000","last_clean":"2024-04-02T16:11:32.169908+0000","last_became_active":"2024-04-02T16:11:32.154518+0000","last_became_peered":"2024-04-02T16:11:32.154518+0000","last_unstale":"2024-04-02T16:11:32.169908+0000","last_undegraded":"2024-04-02T16:11:32.169908+0000","last_fullsized":"2024-04-02T16:11:32.169908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:55.127710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160202+0000","last_change":"2024-04-02T16:11:31.125926+0000","last_active":"2024-04-02T16:11:33.160202+0000","last_peered":"2024-04-02T16:11:33.160202+0000","last_clean":"2024-04-02T16:11:33.160202+0000","last_became_active":"2024-04-02T16:11:31.124109+0000","last_became_peered":"2024-04-02T16:11:31.124109+0000","last_unstale":"2024-04-02T16:11:33.160202+0000","last_undegraded":"2024-04-02T16:11:33.160202+0000","last_fullsized":"2024-04-02T16:11:33.160202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497615+0000","last_change":"2024-04-02T16:11:32.171153+0000","last_active":"2024-04-02T16:11:33.497615+0000","last_peered":"2024-04-02T16:11:33.497615+0000","last_clean":"2024-04-02T16:11:33.497615+0000","last_became_active":"2024-04-02T16:11:32.170399+0000","last_became_peered":"2024-04-02T16:11:32.170399+0000","last_unstale":"2024-04-02T16:11:33.497615+0000","last_undegraded":"2024-04-02T16:11:33.497615+0000","last_fullsized":"2024-04-02T16:11:33.497615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160399+0000","last_change":"2024-04-02T16:11:31.131680+0000","last_active":"2024-04-02T16:11:34.160399+0000","last_peered":"2024-04-02T16:11:34.160399+0000","last_clean":"2024-04-02T16:11:34.160399+0000","last_became_active":"2024-04-02T16:11:31.129432+0000","last_became_peered":"2024-04-02T16:11:31.129432+0000","last_unstale":"2024-04-02T16:11:34.160399+0000","last_undegraded":"2024-04-02T16:11:34.160399+0000","last_fullsized":"2024-04-02T16:11:34.160399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160538+0000","last_change":"2024-04-02T16:11:32.167516+0000","last_active":"2024-04-02T16:11:34.160538+0000","last_peered":"2024-04-02T16:11:34.160538+0000","last_clean":"2024-04-02T16:11:34.160538+0000","last_became_active":"2024-04-02T16:11:32.167201+0000","last_became_peered":"2024-04-02T16:11:32.167201+0000","last_unstale":"2024-04-02T16:11:34.160538+0000","last_undegraded":"2024-04-02T16:11:34.160538+0000","last_fullsized":"2024-04-02T16:11:34.160538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.175668+0000","last_change":"2024-04-02T16:11:31.117868+0000","last_active":"2024-04-02T16:11:32.175668+0000","last_peered":"2024-04-02T16:11:32.175668+0000","last_clean":"2024-04-02T16:11:32.175668+0000","last_became_active":"2024-04-02T16:11:31.117437+0000","last_became_peered":"2024-04-02T16:11:31.117437+0000","last_unstale":"2024-04-02T16:11:32.175668+0000","last_undegraded":"2024-04-02T16:11:32.175668+0000","last_fullsized":"2024-04-02T16:11:32.175668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497088+0000","last_change":"2024-04-02T16:11:32.178110+0000","last_active":"2024-04-02T16:11:33.497088+0000","last_peered":"2024-04-02T16:11:33.497088+0000","last_clean":"2024-04-02T16:11:33.497088+0000","last_became_active":"2024-04-02T16:11:32.177756+0000","last_became_peered":"2024-04-02T16:11:32.177756+0000","last_unstale":"2024-04-02T16:11:33.497088+0000","last_undegraded":"2024-04-02T16:11:33.497088+0000","last_fullsized":"2024-04-02T16:11:33.497088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159238+0000","last_change":"2024-04-02T16:11:31.130795+0000","last_active":"2024-04-02T16:11:33.159238+0000","last_peered":"2024-04-02T16:11:33.159238+0000","last_clean":"2024-04-02T16:11:33.159238+0000","last_became_active":"2024-04-02T16:11:31.129907+0000","last_became_peered":"2024-04-02T16:11:31.129907+0000","last_unstale":"2024-04-02T16:11:33.159238+0000","last_undegraded":"2024-04-02T16:11:33.159238+0000","last_fullsized":"2024-04-02T16:11:33.159238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:27:40.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497755+0000","last_change":"2024-04-02T16:11:32.173519+0000","last_active":"2024-04-02T16:11:33.497755+0000","last_peered":"2024-04-02T16:11:33.497755+0000","last_clean":"2024-04-02T16:11:33.497755+0000","last_became_active":"2024-04-02T16:11:32.173159+0000","last_became_peered":"2024-04-02T16:11:32.173159+0000","last_unstale":"2024-04-02T16:11:33.497755+0000","last_undegraded":"2024-04-02T16:11:33.497755+0000","last_fullsized":"2024-04-02T16:11:33.497755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178348+0000","last_change":"2024-04-02T16:11:31.113477+0000","last_active":"2024-04-02T16:11:32.178348+0000","last_peered":"2024-04-02T16:11:32.178348+0000","last_clean":"2024-04-02T16:11:32.178348+0000","last_became_active":"2024-04-02T16:11:31.112346+0000","last_became_peered":"2024-04-02T16:11:31.112346+0000","last_unstale":"2024-04-02T16:11:32.178348+0000","last_undegraded":"2024-04-02T16:11:32.178348+0000","last_fullsized":"2024-04-02T16:11:32.178348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498157+0000","last_change":"2024-04-02T16:11:32.173602+0000","last_active":"2024-04-02T16:11:33.498157+0000","last_peered":"2024-04-02T16:11:33.498157+0000","last_clean":"2024-04-02T16:11:33.498157+0000","last_became_active":"2024-04-02T16:11:32.173294+0000","last_became_peered":"2024-04-02T16:11:32.173294+0000","last_unstale":"2024-04-02T16:11:33.498157+0000","last_undegraded":"2024-04-02T16:11:33.498157+0000","last_fullsized":"2024-04-02T16:11:33.498157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:50.854339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156803+0000","last_change":"2024-04-02T16:11:31.134514+0000","last_active":"2024-04-02T16:11:33.156803+0000","last_peered":"2024-04-02T16:11:33.156803+0000","last_clean":"2024-04-02T16:11:33.156803+0000","last_became_active":"2024-04-02T16:11:31.134123+0000","last_became_peered":"2024-04-02T16:11:31.134123+0000","last_unstale":"2024-04-02T16:11:33.156803+0000","last_undegraded":"2024-04-02T16:11:33.156803+0000","last_fullsized":"2024-04-02T16:11:33.156803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159396+0000","last_change":"2024-04-02T16:11:32.172101+0000","last_active":"2024-04-02T16:11:33.159396+0000","last_peered":"2024-04-02T16:11:33.159396+0000","last_clean":"2024-04-02T16:11:33.159396+0000","last_became_active":"2024-04-02T16:11:32.169192+0000","last_became_peered":"2024-04-02T16:11:32.169192+0000","last_unstale":"2024-04-02T16:11:33.159396+0000","last_undegraded":"2024-04-02T16:11:33.159396+0000","last_fullsized":"2024-04-02T16:11:33.159396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.133922+0000","last_change":"2024-04-02T16:11:31.118803+0000","last_active":"2024-04-02T16:11:32.133922+0000","last_peered":"2024-04-02T16:11:32.133922+0000","last_clean":"2024-04-02T16:11:32.133922+0000","last_became_active":"2024-04-02T16:11:31.118400+0000","last_became_peered":"2024-04-02T16:11:31.118400+0000","last_unstale":"2024-04-02T16:11:32.133922+0000","last_undegraded":"2024-04-02T16:11:32.133922+0000","last_fullsized":"2024-04-02T16:11:32.133922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497571+0000","last_change":"2024-04-02T16:11:32.175909+0000","last_active":"2024-04-02T16:11:33.497571+0000","last_peered":"2024-04-02T16:11:33.497571+0000","last_clean":"2024-04-02T16:11:33.497571+0000","last_became_active":"2024-04-02T16:11:32.175629+0000","last_became_peered":"2024-04-02T16:11:32.175629+0000","last_unstale":"2024-04-02T16:11:33.497571+0000","last_undegraded":"2024-04-02T16:11:33.497571+0000","last_fullsized":"2024-04-02T16:11:33.497571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496798+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:11:33.496798+0000","last_peered":"2024-04-02T16:11:33.496798+0000","last_clean":"2024-04-02T16:11:33.496798+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:11:33.496798+0000","last_undegraded":"2024-04-02T16:11:33.496798+0000","last_fullsized":"2024-04-02T16:11:33.496798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158329+0000","last_change":"2024-04-02T16:11:31.114855+0000","last_active":"2024-04-02T16:11:33.158329+0000","last_peered":"2024-04-02T16:11:33.158329+0000","last_clean":"2024-04-02T16:11:33.158329+0000","last_became_active":"2024-04-02T16:11:31.114136+0000","last_became_peered":"2024-04-02T16:11:31.114136+0000","last_unstale":"2024-04-02T16:11:33.158329+0000","last_undegraded":"2024-04-02T16:11:33.158329+0000","last_fullsized":"2024-04-02T16:11:33.158329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179205+0000","last_change":"2024-04-02T16:11:32.177918+0000","last_active":"2024-04-02T16:11:32.179205+0000","last_peered":"2024-04-02T16:11:32.179205+0000","last_clean":"2024-04-02T16:11:32.179205+0000","last_became_active":"2024-04-02T16:11:32.177523+0000","last_became_peered":"2024-04-02T16:11:32.177523+0000","last_unstale":"2024-04-02T16:11:32.179205+0000","last_undegraded":"2024-04-02T16:11:32.179205+0000","last_fullsized":"2024-04-02T16:11:32.179205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:27.528611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496680+0000","last_change":"2024-04-02T16:11:31.147127+0000","last_active":"2024-04-02T16:11:33.496680+0000","last_peered":"2024-04-02T16:11:33.496680+0000","last_clean":"2024-04-02T16:11:33.496680+0000","last_became_active":"2024-04-02T16:11:31.146323+0000","last_became_peered":"2024-04-02T16:11:31.146323+0000","last_unstale":"2024-04-02T16:11:33.496680+0000","last_undegraded":"2024-04-02T16:11:33.496680+0000","last_fullsized":"2024-04-02T16:11:33.496680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158507+0000","last_change":"2024-04-02T16:11:32.152084+0000","last_active":"2024-04-02T16:11:32.158507+0000","last_peered":"2024-04-02T16:11:32.158507+0000","last_clean":"2024-04-02T16:11:32.158507+0000","last_became_active":"2024-04-02T16:11:32.151565+0000","last_became_peered":"2024-04-02T16:11:32.151565+0000","last_unstale":"2024-04-02T16:11:32.158507+0000","last_undegraded":"2024-04-02T16:11:32.158507+0000","last_fullsized":"2024-04-02T16:11:32.158507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.134373+0000","last_change":"2024-04-02T16:11:31.112830+0000","last_active":"2024-04-02T16:11:32.134373+0000","last_peered":"2024-04-02T16:11:32.134373+0000","last_clean":"2024-04-02T16:11:32.134373+0000","last_became_active":"2024-04-02T16:11:31.112472+0000","last_became_peered":"2024-04-02T16:11:31.112472+0000","last_unstale":"2024-04-02T16:11:32.134373+0000","last_undegraded":"2024-04-02T16:11:32.134373+0000","last_fullsized":"2024-04-02T16:11:32.134373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156504+0000","last_change":"2024-04-02T16:11:32.152287+0000","last_active":"2024-04-02T16:11:33.156504+0000","last_peered":"2024-04-02T16:11:33.156504+0000","last_clean":"2024-04-02T16:11:33.156504+0000","last_became_active":"2024-04-02T16:11:32.151559+0000","last_became_peered":"2024-04-02T16:11:32.151559+0000","last_unstale":"2024-04-02T16:11:33.156504+0000","last_undegraded":"2024-04-02T16:11:33.156504+0000","last_fullsized":"2024-04-02T16:11:33.156504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158997+0000","last_change":"2024-04-02T16:11:31.131581+0000","last_active":"2024-04-02T16:11:33.158997+0000","last_peered":"2024-04-02T16:11:33.158997+0000","last_clean":"2024-04-02T16:11:33.158997+0000","last_became_active":"2024-04-02T16:11:31.130730+0000","last_became_peered":"2024-04-02T16:11:31.130730+0000","last_unstale":"2024-04-02T16:11:33.158997+0000","last_undegraded":"2024-04-02T16:11:33.158997+0000","last_fullsized":"2024-04-02T16:11:33.158997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157459+0000","last_change":"2024-04-02T16:11:32.159434+0000","last_active":"2024-04-02T16:11:33.157459+0000","last_peered":"2024-04-02T16:11:33.157459+0000","last_clean":"2024-04-02T16:11:33.157459+0000","last_became_active":"2024-04-02T16:11:32.159056+0000","last_became_peered":"2024-04-02T16:11:32.159056+0000","last_unstale":"2024-04-02T16:11:33.157459+0000","last_undegraded":"2024-04-02T16:11:33.157459+0000","last_fullsized":"2024-04-02T16:11:33.157459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156640+0000","last_change":"2024-04-02T16:11:31.131364+0000","last_active":"2024-04-02T16:11:33.156640+0000","last_peered":"2024-04-02T16:11:33.156640+0000","last_clean":"2024-04-02T16:11:33.156640+0000","last_became_active":"2024-04-02T16:11:31.130814+0000","last_became_peered":"2024-04-02T16:11:31.130814+0000","last_unstale":"2024-04-02T16:11:33.156640+0000","last_undegraded":"2024-04-02T16:11:33.156640+0000","last_fullsized":"2024-04-02T16:11:33.156640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160072+0000","last_change":"2024-04-02T16:11:32.168617+0000","last_active":"2024-04-02T16:11:33.160072+0000","last_peered":"2024-04-02T16:11:33.160072+0000","last_clean":"2024-04-02T16:11:33.160072+0000","last_became_active":"2024-04-02T16:11:32.168318+0000","last_became_peered":"2024-04-02T16:11:32.168318+0000","last_unstale":"2024-04-02T16:11:33.160072+0000","last_undegraded":"2024-04-02T16:11:33.160072+0000","last_fullsized":"2024-04-02T16:11:33.160072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497910+0000","last_change":"2024-04-02T16:11:31.150504+0000","last_active":"2024-04-02T16:11:33.497910+0000","last_peered":"2024-04-02T16:11:33.497910+0000","last_clean":"2024-04-02T16:11:33.497910+0000","last_became_active":"2024-04-02T16:11:31.149962+0000","last_became_peered":"2024-04-02T16:11:31.149962+0000","last_unstale":"2024-04-02T16:11:33.497910+0000","last_undegraded":"2024-04-02T16:11:33.497910+0000","last_fullsized":"2024-04-02T16:11:33.497910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:35.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156819+0000","last_change":"2024-04-02T16:11:32.157220+0000","last_active":"2024-04-02T16:11:33.156819+0000","last_peered":"2024-04-02T16:11:33.156819+0000","last_clean":"2024-04-02T16:11:33.156819+0000","last_became_active":"2024-04-02T16:11:32.156725+0000","last_became_peered":"2024-04-02T16:11:32.156725+0000","last_unstale":"2024-04-02T16:11:33.156819+0000","last_undegraded":"2024-04-02T16:11:33.156819+0000","last_fullsized":"2024-04-02T16:11:33.156819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.159254+0000","last_change":"2024-04-02T16:11:31.130016+0000","last_active":"2024-04-02T16:11:34.159254+0000","last_peered":"2024-04-02T16:11:34.159254+0000","last_clean":"2024-04-02T16:11:34.159254+0000","last_became_active":"2024-04-02T16:11:31.129578+0000","last_became_peered":"2024-04-02T16:11:31.129578+0000","last_unstale":"2024-04-02T16:11:34.159254+0000","last_undegraded":"2024-04-02T16:11:34.159254+0000","last_fullsized":"2024-04-02T16:11:34.159254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.165198+0000","last_change":"2024-04-02T16:11:32.162159+0000","last_active":"2024-04-02T16:11:32.165198+0000","last_peered":"2024-04-02T16:11:32.165198+0000","last_clean":"2024-04-02T16:11:32.165198+0000","last_became_active":"2024-04-02T16:11:32.161875+0000","last_became_peered":"2024-04-02T16:11:32.161875+0000","last_unstale":"2024-04-02T16:11:32.165198+0000","last_undegraded":"2024-04-02T16:11:32.165198+0000","last_fullsized":"2024-04-02T16:11:32.165198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161008+0000","last_change":"2024-04-02T16:11:31.143750+0000","last_active":"2024-04-02T16:11:34.161008+0000","last_peered":"2024-04-02T16:11:34.161008+0000","last_clean":"2024-04-02T16:11:34.161008+0000","last_became_active":"2024-04-02T16:11:31.142991+0000","last_became_peered":"2024-04-02T16:11:31.142991+0000","last_unstale":"2024-04-02T16:11:34.161008+0000","last_undegraded":"2024-04-02T16:11:34.161008+0000","last_fullsized":"2024-04-02T16:11:34.161008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.161375+0000","last_change":"2024-04-02T16:11:32.153539+0000","last_active":"2024-04-02T16:11:32.161375+0000","last_peered":"2024-04-02T16:11:32.161375+0000","last_clean":"2024-04-02T16:11:32.161375+0000","last_became_active":"2024-04-02T16:11:32.152769+0000","last_became_peered":"2024-04-02T16:11:32.152769+0000","last_unstale":"2024-04-02T16:11:32.161375+0000","last_undegraded":"2024-04-02T16:11:32.161375+0000","last_fullsized":"2024-04-02T16:11:32.161375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496646+0000","last_change":"2024-04-02T16:11:31.144365+0000","last_active":"2024-04-02T16:11:33.496646+0000","last_peered":"2024-04-02T16:11:33.496646+0000","last_clean":"2024-04-02T16:11:33.496646+0000","last_became_active":"2024-04-02T16:11:31.143659+0000","last_became_peered":"2024-04-02T16:11:31.143659+0000","last_unstale":"2024-04-02T16:11:33.496646+0000","last_undegraded":"2024-04-02T16:11:33.496646+0000","last_fullsized":"2024-04-02T16:11:33.496646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157111+0000","last_change":"2024-04-02T16:11:32.153561+0000","last_active":"2024-04-02T16:11:33.157111+0000","last_peered":"2024-04-02T16:11:33.157111+0000","last_clean":"2024-04-02T16:11:33.157111+0000","last_became_active":"2024-04-02T16:11:32.148826+0000","last_became_peered":"2024-04-02T16:11:32.148826+0000","last_unstale":"2024-04-02T16:11:33.157111+0000","last_undegraded":"2024-04-02T16:11:33.157111+0000","last_fullsized":"2024-04-02T16:11:33.157111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156589+0000","last_change":"2024-04-02T16:11:31.144769+0000","last_active":"2024-04-02T16:11:33.156589+0000","last_peered":"2024-04-02T16:11:33.156589+0000","last_clean":"2024-04-02T16:11:33.156589+0000","last_became_active":"2024-04-02T16:11:31.144386+0000","last_became_peered":"2024-04-02T16:11:31.144386+0000","last_unstale":"2024-04-02T16:11:33.156589+0000","last_undegraded":"2024-04-02T16:11:33.156589+0000","last_fullsized":"2024-04-02T16:11:33.156589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.164266+0000","last_change":"2024-04-02T16:11:32.160650+0000","last_active":"2024-04-02T16:11:32.164266+0000","last_peered":"2024-04-02T16:11:32.164266+0000","last_clean":"2024-04-02T16:11:32.164266+0000","last_became_active":"2024-04-02T16:11:32.160239+0000","last_became_peered":"2024-04-02T16:11:32.160239+0000","last_unstale":"2024-04-02T16:11:32.164266+0000","last_undegraded":"2024-04-02T16:11:32.164266+0000","last_fullsized":"2024-04-02T16:11:32.164266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:23.736628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157210+0000","last_change":"2024-04-02T16:11:31.119214+0000","last_active":"2024-04-02T16:11:33.157210+0000","last_peered":"2024-04-02T16:11:33.157210+0000","last_clean":"2024-04-02T16:11:33.157210+0000","last_became_active":"2024-04-02T16:11:31.118885+0000","last_became_peered":"2024-04-02T16:11:31.118885+0000","last_unstale":"2024-04-02T16:11:33.157210+0000","last_undegraded":"2024-04-02T16:11:33.157210+0000","last_fullsized":"2024-04-02T16:11:33.157210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.177821+0000","last_change":"2024-04-02T16:11:32.165991+0000","last_active":"2024-04-02T16:11:32.177821+0000","last_peered":"2024-04-02T16:11:32.177821+0000","last_clean":"2024-04-02T16:11:32.177821+0000","last_became_active":"2024-04-02T16:11:32.165617+0000","last_became_peered":"2024-04-02T16:11:32.165617+0000","last_unstale":"2024-04-02T16:11:32.177821+0000","last_undegraded":"2024-04-02T16:11:32.177821+0000","last_fullsized":"2024-04-02T16:11:32.177821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497809+0000","last_change":"2024-04-02T16:11:31.150660+0000","last_active":"2024-04-02T16:11:33.497809+0000","last_peered":"2024-04-02T16:11:33.497809+0000","last_clean":"2024-04-02T16:11:33.497809+0000","last_became_active":"2024-04-02T16:11:31.149922+0000","last_became_peered":"2024-04-02T16:11:31.149922+0000","last_unstale":"2024-04-02T16:11:33.497809+0000","last_undegraded":"2024-04-02T16:11:33.497809+0000","last_fullsized":"2024-04-02T16:11:33.497809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:54.118266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160589+0000","last_change":"2024-04-02T16:11:32.166809+0000","last_active":"2024-04-02T16:11:34.160589+0000","last_peered":"2024-04-02T16:11:34.160589+0000","last_clean":"2024-04-02T16:11:34.160589+0000","last_became_active":"2024-04-02T16:11:32.164926+0000","last_became_peered":"2024-04-02T16:11:32.164926+0000","last_unstale":"2024-04-02T16:11:34.160589+0000","last_undegraded":"2024-04-02T16:11:34.160589+0000","last_fullsized":"2024-04-02T16:11:34.160589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159873+0000","last_change":"2024-04-02T16:11:32.146387+0000","last_active":"2024-04-02T16:11:33.159873+0000","last_peered":"2024-04-02T16:11:33.159873+0000","last_clean":"2024-04-02T16:11:33.159873+0000","last_became_active":"2024-04-02T16:11:32.145655+0000","last_became_peered":"2024-04-02T16:11:32.145655+0000","last_unstale":"2024-04-02T16:11:33.159873+0000","last_undegraded":"2024-04-02T16:11:33.159873+0000","last_fullsized":"2024-04-02T16:11:33.159873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497313+0000","last_change":"2024-04-02T16:11:31.147426+0000","last_active":"2024-04-02T16:11:33.497313+0000","last_peered":"2024-04-02T16:11:33.497313+0000","last_clean":"2024-04-02T16:11:33.497313+0000","last_became_active":"2024-04-02T16:11:31.146661+0000","last_became_peered":"2024-04-02T16:11:31.146661+0000","last_unstale":"2024-04-02T16:11:33.497313+0000","last_undegraded":"2024-04-02T16:11:33.497313+0000","last_fullsized":"2024-04-02T16:11:33.497313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159146+0000","last_change":"2024-04-02T16:11:32.163890+0000","last_active":"2024-04-02T16:11:33.159146+0000","last_peered":"2024-04-02T16:11:33.159146+0000","last_clean":"2024-04-02T16:11:33.159146+0000","last_became_active":"2024-04-02T16:11:32.163585+0000","last_became_peered":"2024-04-02T16:11:32.163585+0000","last_unstale":"2024-04-02T16:11:33.159146+0000","last_undegraded":"2024-04-02T16:11:33.159146+0000","last_fullsized":"2024-04-02T16:11:33.159146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497629+0000","last_change":"2024-04-02T16:11:31.151627+0000","last_active":"2024-04-02T16:11:33.497629+0000","last_peered":"2024-04-02T16:11:33.497629+0000","last_clean":"2024-04-02T16:11:33.497629+0000","last_became_active":"2024-04-02T16:11:31.151077+0000","last_became_peered":"2024-04-02T16:11:31.151077+0000","last_unstale":"2024-04-02T16:11:33.497629+0000","last_undegraded":"2024-04-02T16:11:33.497629+0000","last_fullsized":"2024-04-02T16:11:33.497629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:47.686560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158552+0000","last_change":"2024-04-02T16:11:32.153198+0000","last_active":"2024-04-02T16:11:32.158552+0000","last_peered":"2024-04-02T16:11:32.158552+0000","last_clean":"2024-04-02T16:11:32.158552+0000","last_became_active":"2024-04-02T16:11:32.149403+0000","last_became_peered":"2024-04-02T16:11:32.149403+0000","last_unstale":"2024-04-02T16:11:32.158552+0000","last_undegraded":"2024-04-02T16:11:32.158552+0000","last_fullsized":"2024-04-02T16:11:32.158552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178419+0000","last_change":"2024-04-02T16:11:31.128936+0000","last_active":"2024-04-02T16:11:32.178419+0000","last_peered":"2024-04-02T16:11:32.178419+0000","last_clean":"2024-04-02T16:11:32.178419+0000","last_became_active":"2024-04-02T16:11:31.128574+0000","last_became_peered":"2024-04-02T16:11:31.128574+0000","last_unstale":"2024-04-02T16:11:32.178419+0000","last_undegraded":"2024-04-02T16:11:32.178419+0000","last_fullsized":"2024-04-02T16:11:32.178419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178613+0000","last_change":"2024-04-02T16:11:32.170441+0000","last_active":"2024-04-02T16:11:32.178613+0000","last_peered":"2024-04-02T16:11:32.178613+0000","last_clean":"2024-04-02T16:11:32.178613+0000","last_became_active":"2024-04-02T16:11:32.167070+0000","last_became_peered":"2024-04-02T16:11:32.167070+0000","last_unstale":"2024-04-02T16:11:32.178613+0000","last_undegraded":"2024-04-02T16:11:32.178613+0000","last_fullsized":"2024-04-02T16:11:32.178613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.174093+0000","last_change":"2024-04-02T16:11:31.142814+0000","last_active":"2024-04-02T16:11:32.174093+0000","last_peered":"2024-04-02T16:11:32.174093+0000","last_clean":"2024-04-02T16:11:32.174093+0000","last_became_active":"2024-04-02T16:11:31.142568+0000","last_became_peered":"2024-04-02T16:11:31.142568+0000","last_unstale":"2024-04-02T16:11:32.174093+0000","last_undegraded":"2024-04-02T16:11:32.174093+0000","last_fullsized":"2024-04-02T16:11:32.174093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:11:36.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:11:36.337 INFO:tasks.ceph.mds.b.smithi204.stderr:2024-04-02T16:11:36.337+0000 7f366df73640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:11:36.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 Processor -- start 2024-04-02T16:11:36.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 -- start start 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb600b3500 0x7fbb600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600b6310 con 0x7fbb600b8c80 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600b55d0 con 0x7fbb600b4bf0 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb65e36640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6014b560 con 0x7fbb600b3500 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb600b3500 0x7fbb600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb600b3500 0x7fbb600b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37020/0 (socket says 172.21.15.40:37020) 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 -- 172.21.15.40:0/2379229049 learned_addr learned my addr 172.21.15.40:0/2379229049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5ffff640 1 --1- 172.21.15.40:0/2379229049 >> v1:172.21.15.40:6789/0 conn(0x7fbb600b8c80 0x7fbb601521e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50800/0 (socket says 172.21.15.40:50800) 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 -- 172.21.15.40:0/2379229049 >> v1:172.21.15.204:6789/0 conn(0x7fbb600b4bf0 legacy=0x7fbb6014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 -- 172.21.15.40:0/2379229049 >> v1:172.21.15.40:6789/0 conn(0x7fbb600b8c80 legacy=0x7fbb601521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.361+0000 7fbb5effd640 1 -- 172.21.15.40:0/2379229049 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600b3fa0 con 0x7fbb600b3500 2024-04-02T16:11:36.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5effd640 1 --2- 172.21.15.40:0/2379229049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b3500 0x7fbb600b4fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009e50 tx=0x7fbb5401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a60f5759dd0063 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5e7fc640 1 -- 172.21.15.40:0/2379229049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540a5590 con 0x7fbb600b3500 2024-04-02T16:11:36.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5e7fc640 1 -- 172.21.15.40:0/2379229049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb540a5730 con 0x7fbb600b3500 2024-04-02T16:11:36.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5e7fc640 1 -- 172.21.15.40:0/2379229049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb540a5b00 con 0x7fbb600b3500 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/2379229049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b3500 msgr2=0x7fbb600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 --2- 172.21.15.40:0/2379229049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b3500 0x7fbb600b4fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009e50 tx=0x7fbb5401c8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/2379229049 shutdown_connections 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 --2- 172.21.15.40:0/2379229049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b3500 0x7fbb600b4fd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/2379229049 >> 172.21.15.40:0/2379229049 conn(0x7fbb600734b0 msgr2=0x7fbb600bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/2379229049 shutdown_connections 2024-04-02T16:11:36.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/2379229049 wait complete. 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 Processor -- start 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- start start 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb600b3500 0x7fbb6013c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb600b4bf0 0x7fbb601e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60156740 con 0x7fbb600b8c80 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601568c0 con 0x7fbb600b4bf0 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb60156a40 con 0x7fbb600b3500 2024-04-02T16:11:36.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37022/0 (socket says 172.21.15.40:37022) 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 -- 172.21.15.40:0/3509453971 learned_addr learned my addr 172.21.15.40:0/3509453971 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5effd640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb600b4bf0 0x7fbb601e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5f7fe640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb600b3500 0x7fbb6013c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 -- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb600b3500 msgr2=0x7fbb6013c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb600b3500 0x7fbb6013c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 -- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb600b4bf0 msgr2=0x7fbb601e0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb600b4bf0 0x7fbb601e0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 -- 172.21.15.40:0/3509453971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6013aba0 con 0x7fbb600b8c80 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb5ffff640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c390 tx=0x7fbb5800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580074e0 con 0x7fbb600b8c80 2024-04-02T16:11:36.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58007680 con 0x7fbb600b8c80 2024-04-02T16:11:36.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb60145f70 con 0x7fbb600b8c80 2024-04-02T16:11:36.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58004a10 con 0x7fbb600b8c80 2024-04-02T16:11:36.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.365+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6013ae40 con 0x7fbb600b8c80 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6013b080 con 0x7fbb600b8c80 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbb5801a020 con 0x7fbb600b8c80 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb47fff640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 0x7fbb4c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb580860c0 con 0x7fbb600b8c80 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb5f7fe640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 0x7fbb4c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.369+0000 7fbb5f7fe640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 0x7fbb4c06a870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b810 tx=0x7fbb50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.373+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fbb6013b080 con 0x7fbb600b8c80 2024-04-02T16:11:36.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.533+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb600685f0 con 0x7fbb4c0683b0 2024-04-02T16:11:36.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.549+0000 7fbb47fff640 1 -- 172.21.15.40:0/3509453971 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251919 (secure 0 0 0) 0x7fbb600685f0 con 0x7fbb4c0683b0 2024-04-02T16:11:36.557 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:36.559 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 msgr2=0x7fbb4c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 0x7fbb4c06a870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b810 tx=0x7fbb50008040 comp rx=0 tx=0).stop 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 msgr2=0x7fbb601e1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c390 tx=0x7fbb5800c850 comp rx=0 tx=0).stop 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 shutdown_connections 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbb4c0683b0 0x7fbb4c06a870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb600b8c80 0x7fbb601e1420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb600b4bf0 0x7fbb601e0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 --2- 172.21.15.40:0/3509453971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb600b3500 0x7fbb6013c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 >> 172.21.15.40:0/3509453971 conn(0x7fbb600734b0 msgr2=0x7fbb600baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 shutdown_connections 2024-04-02T16:11:36.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.553+0000 7fbb65e36640 1 -- 172.21.15.40:0/3509453971 wait complete. 2024-04-02T16:11:36.576 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-02T16:11:35.848781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218972,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755748,"statfs":{"total":773094113280,"available":772869885952,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753044"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497949+0000","last_change":"2024-04-02T16:11:32.176361+0000","last_active":"2024-04-02T16:11:33.497949+0000","last_peered":"2024-04-02T16:11:33.497949+0000","last_clean":"2024-04-02T16:11:33.497949+0000","last_became_active":"2024-04-02T16:11:32.176117+0000","last_became_peered":"2024-04-02T16:11:32.176117+0000","last_unstale":"2024-04-02T16:11:33.497949+0000","last_undegraded":"2024-04-02T16:11:33.497949+0000","last_fullsized":"2024-04-02T16:11:33.497949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:03.664122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156872+0000","last_change":"2024-04-02T16:11:32.165208+0000","last_active":"2024-04-02T16:11:33.156872+0000","last_peered":"2024-04-02T16:11:33.156872+0000","last_clean":"2024-04-02T16:11:33.156872+0000","last_became_active":"2024-04-02T16:11:32.164869+0000","last_became_peered":"2024-04-02T16:11:32.164869+0000","last_unstale":"2024-04-02T16:11:33.156872+0000","last_undegraded":"2024-04-02T16:11:33.156872+0000","last_fullsized":"2024-04-02T16:11:33.156872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158555+0000","last_change":"2024-04-02T16:11:32.146435+0000","last_active":"2024-04-02T16:11:32.158555+0000","last_peered":"2024-04-02T16:11:32.158555+0000","last_clean":"2024-04-02T16:11:32.158555+0000","last_became_active":"2024-04-02T16:11:32.145704+0000","last_became_peered":"2024-04-02T16:11:32.145704+0000","last_unstale":"2024-04-02T16:11:32.158555+0000","last_undegraded":"2024-04-02T16:11:32.158555+0000","last_fullsized":"2024-04-02T16:11:32.158555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:44.133779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158980+0000","last_change":"2024-04-02T16:11:32.172527+0000","last_active":"2024-04-02T16:11:33.158980+0000","last_peered":"2024-04-02T16:11:33.158980+0000","last_clean":"2024-04-02T16:11:33.158980+0000","last_became_active":"2024-04-02T16:11:32.170021+0000","last_became_peered":"2024-04-02T16:11:32.170021+0000","last_unstale":"2024-04-02T16:11:33.158980+0000","last_undegraded":"2024-04-02T16:11:33.158980+0000","last_fullsized":"2024-04-02T16:11:33.158980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497274+0000","last_change":"2024-04-02T16:11:32.171344+0000","last_active":"2024-04-02T16:11:33.497274+0000","last_peered":"2024-04-02T16:11:33.497274+0000","last_clean":"2024-04-02T16:11:33.497274+0000","last_became_active":"2024-04-02T16:11:32.170838+0000","last_became_peered":"2024-04-02T16:11:32.170838+0000","last_unstale":"2024-04-02T16:11:33.497274+0000","last_undegraded":"2024-04-02T16:11:33.497274+0000","last_fullsized":"2024-04-02T16:11:33.497274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158169+0000","last_change":"2024-04-02T16:11:32.160886+0000","last_active":"2024-04-02T16:11:33.158169+0000","last_peered":"2024-04-02T16:11:33.158169+0000","last_clean":"2024-04-02T16:11:33.158169+0000","last_became_active":"2024-04-02T16:11:32.159998+0000","last_became_peered":"2024-04-02T16:11:32.159998+0000","last_unstale":"2024-04-02T16:11:33.158169+0000","last_undegraded":"2024-04-02T16:11:33.158169+0000","last_fullsized":"2024-04-02T16:11:33.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496848+0000","last_change":"2024-04-02T16:11:32.172231+0000","last_active":"2024-04-02T16:11:33.496848+0000","last_peered":"2024-04-02T16:11:33.496848+0000","last_clean":"2024-04-02T16:11:33.496848+0000","last_became_active":"2024-04-02T16:11:32.171852+0000","last_became_peered":"2024-04-02T16:11:32.171852+0000","last_unstale":"2024-04-02T16:11:33.496848+0000","last_undegraded":"2024-04-02T16:11:33.496848+0000","last_fullsized":"2024-04-02T16:11:33.496848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178509+0000","last_change":"2024-04-02T16:11:32.171342+0000","last_active":"2024-04-02T16:11:32.178509+0000","last_peered":"2024-04-02T16:11:32.178509+0000","last_clean":"2024-04-02T16:11:32.178509+0000","last_became_active":"2024-04-02T16:11:32.171027+0000","last_became_peered":"2024-04-02T16:11:32.171027+0000","last_unstale":"2024-04-02T16:11:32.178509+0000","last_undegraded":"2024-04-02T16:11:32.178509+0000","last_fullsized":"2024-04-02T16:11:32.178509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:55.127710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498216+0000","last_change":"2024-04-02T16:11:32.171461+0000","last_active":"2024-04-02T16:11:33.498216+0000","last_peered":"2024-04-02T16:11:33.498216+0000","last_clean":"2024-04-02T16:11:33.498216+0000","last_became_active":"2024-04-02T16:11:32.170971+0000","last_became_peered":"2024-04-02T16:11:32.170971+0000","last_unstale":"2024-04-02T16:11:33.498216+0000","last_undegraded":"2024-04-02T16:11:33.498216+0000","last_fullsized":"2024-04-02T16:11:33.498216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:25.000722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161307+0000","last_change":"2024-04-02T16:11:32.164006+0000","last_active":"2024-04-02T16:11:34.161307+0000","last_peered":"2024-04-02T16:11:34.161307+0000","last_clean":"2024-04-02T16:11:34.161307+0000","last_became_active":"2024-04-02T16:11:32.163712+0000","last_became_peered":"2024-04-02T16:11:32.163712+0000","last_unstale":"2024-04-02T16:11:34.161307+0000","last_undegraded":"2024-04-02T16:11:34.161307+0000","last_fullsized":"2024-04-02T16:11:34.161307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160963+0000","last_change":"2024-04-02T16:11:32.162643+0000","last_active":"2024-04-02T16:11:34.160963+0000","last_peered":"2024-04-02T16:11:34.160963+0000","last_clean":"2024-04-02T16:11:34.160963+0000","last_became_active":"2024-04-02T16:11:32.162329+0000","last_became_peered":"2024-04-02T16:11:32.162329+0000","last_unstale":"2024-04-02T16:11:34.160963+0000","last_undegraded":"2024-04-02T16:11:34.160963+0000","last_fullsized":"2024-04-02T16:11:34.160963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158378+0000","last_change":"2024-04-02T16:11:32.150507+0000","last_active":"2024-04-02T16:11:33.158378+0000","last_peered":"2024-04-02T16:11:33.158378+0000","last_clean":"2024-04-02T16:11:33.158378+0000","last_became_active":"2024-04-02T16:11:32.149817+0000","last_became_peered":"2024-04-02T16:11:32.149817+0000","last_unstale":"2024-04-02T16:11:33.158378+0000","last_undegraded":"2024-04-02T16:11:33.158378+0000","last_fullsized":"2024-04-02T16:11:33.158378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178417+0000","last_change":"2024-04-02T16:11:32.171214+0000","last_active":"2024-04-02T16:11:32.178417+0000","last_peered":"2024-04-02T16:11:32.178417+0000","last_clean":"2024-04-02T16:11:32.178417+0000","last_became_active":"2024-04-02T16:11:32.170528+0000","last_became_peered":"2024-04-02T16:11:32.170528+0000","last_unstale":"2024-04-02T16:11:32.178417+0000","last_undegraded":"2024-04-02T16:11:32.178417+0000","last_fullsized":"2024-04-02T16:11:32.178417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158518+0000","last_change":"2024-04-02T16:11:32.148238+0000","last_active":"2024-04-02T16:11:32.158518+0000","last_peered":"2024-04-02T16:11:32.158518+0000","last_clean":"2024-04-02T16:11:32.158518+0000","last_became_active":"2024-04-02T16:11:32.147348+0000","last_became_peered":"2024-04-02T16:11:32.147348+0000","last_unstale":"2024-04-02T16:11:32.158518+0000","last_undegraded":"2024-04-02T16:11:32.158518+0000","last_fullsized":"2024-04-02T16:11:32.158518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:23.736628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156485+0000","last_change":"2024-04-02T16:11:32.174912+0000","last_active":"2024-04-02T16:11:33.156485+0000","last_peered":"2024-04-02T16:11:33.156485+0000","last_clean":"2024-04-02T16:11:33.156485+0000","last_became_active":"2024-04-02T16:11:32.174581+0000","last_became_peered":"2024-04-02T16:11:32.174581+0000","last_unstale":"2024-04-02T16:11:33.156485+0000","last_undegraded":"2024-04-02T16:11:33.156485+0000","last_fullsized":"2024-04-02T16:11:33.156485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156962+0000","last_change":"2024-04-02T16:11:32.160602+0000","last_active":"2024-04-02T16:11:33.156962+0000","last_peered":"2024-04-02T16:11:33.156962+0000","last_clean":"2024-04-02T16:11:33.156962+0000","last_became_active":"2024-04-02T16:11:32.160289+0000","last_became_peered":"2024-04-02T16:11:32.160289+0000","last_unstale":"2024-04-02T16:11:33.156962+0000","last_undegraded":"2024-04-02T16:11:33.156962+0000","last_fullsized":"2024-04-02T16:11:33.156962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159486+0000","last_change":"2024-04-02T16:11:32.159580+0000","last_active":"2024-04-02T16:11:33.159486+0000","last_peered":"2024-04-02T16:11:33.159486+0000","last_clean":"2024-04-02T16:11:33.159486+0000","last_became_active":"2024-04-02T16:11:32.159248+0000","last_became_peered":"2024-04-02T16:11:32.159248+0000","last_unstale":"2024-04-02T16:11:33.159486+0000","last_undegraded":"2024-04-02T16:11:33.159486+0000","last_fullsized":"2024-04-02T16:11:33.159486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.156055+0000","last_change":"2024-04-02T16:11:32.152173+0000","last_active":"2024-04-02T16:11:32.156055+0000","last_peered":"2024-04-02T16:11:32.156055+0000","last_clean":"2024-04-02T16:11:32.156055+0000","last_became_active":"2024-04-02T16:11:32.150901+0000","last_became_peered":"2024-04-02T16:11:32.150901+0000","last_unstale":"2024-04-02T16:11:32.156055+0000","last_undegraded":"2024-04-02T16:11:32.156055+0000","last_fullsized":"2024-04-02T16:11:32.156055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497849+0000","last_change":"2024-04-02T16:11:32.174590+0000","last_active":"2024-04-02T16:11:33.497849+0000","last_peered":"2024-04-02T16:11:33.497849+0000","last_clean":"2024-04-02T16:11:33.497849+0000","last_became_active":"2024-04-02T16:11:32.174269+0000","last_became_peered":"2024-04-02T16:11:32.174269+0000","last_unstale":"2024-04-02T16:11:33.497849+0000","last_undegraded":"2024-04-02T16:11:33.497849+0000","last_fullsized":"2024-04-02T16:11:33.497849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160975+0000","last_change":"2024-04-02T16:11:32.174103+0000","last_active":"2024-04-02T16:11:34.160975+0000","last_peered":"2024-04-02T16:11:34.160975+0000","last_clean":"2024-04-02T16:11:34.160975+0000","last_became_active":"2024-04-02T16:11:32.173844+0000","last_became_peered":"2024-04-02T16:11:32.173844+0000","last_unstale":"2024-04-02T16:11:34.160975+0000","last_undegraded":"2024-04-02T16:11:34.160975+0000","last_fullsized":"2024-04-02T16:11:34.160975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.170281+0000","last_change":"2024-04-02T16:11:32.155248+0000","last_active":"2024-04-02T16:11:32.170281+0000","last_peered":"2024-04-02T16:11:32.170281+0000","last_clean":"2024-04-02T16:11:32.170281+0000","last_became_active":"2024-04-02T16:11:32.154493+0000","last_became_peered":"2024-04-02T16:11:32.154493+0000","last_unstale":"2024-04-02T16:11:32.170281+0000","last_undegraded":"2024-04-02T16:11:32.170281+0000","last_fullsized":"2024-04-02T16:11:32.170281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158188+0000","last_change":"2024-04-02T16:11:32.163843+0000","last_active":"2024-04-02T16:11:33.158188+0000","last_peered":"2024-04-02T16:11:33.158188+0000","last_clean":"2024-04-02T16:11:33.158188+0000","last_became_active":"2024-04-02T16:11:32.163493+0000","last_became_peered":"2024-04-02T16:11:32.163493+0000","last_unstale":"2024-04-02T16:11:33.158188+0000","last_undegraded":"2024-04-02T16:11:33.158188+0000","last_fullsized":"2024-04-02T16:11:33.158188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.166023+0000","last_change":"2024-04-02T16:11:32.160644+0000","last_active":"2024-04-02T16:11:32.166023+0000","last_peered":"2024-04-02T16:11:32.166023+0000","last_clean":"2024-04-02T16:11:32.166023+0000","last_became_active":"2024-04-02T16:11:32.160265+0000","last_became_peered":"2024-04-02T16:11:32.160265+0000","last_unstale":"2024-04-02T16:11:32.166023+0000","last_undegraded":"2024-04-02T16:11:32.166023+0000","last_fullsized":"2024-04-02T16:11:32.166023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:44.133779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157349+0000","last_change":"2024-04-02T16:11:32.169639+0000","last_active":"2024-04-02T16:11:33.157349+0000","last_peered":"2024-04-02T16:11:33.157349+0000","last_clean":"2024-04-02T16:11:33.157349+0000","last_became_active":"2024-04-02T16:11:32.166290+0000","last_became_peered":"2024-04-02T16:11:32.166290+0000","last_unstale":"2024-04-02T16:11:33.157349+0000","last_undegraded":"2024-04-02T16:11:33.157349+0000","last_fullsized":"2024-04-02T16:11:33.157349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179020+0000","last_change":"2024-04-02T16:11:32.172667+0000","last_active":"2024-04-02T16:11:32.179020+0000","last_peered":"2024-04-02T16:11:32.179020+0000","last_clean":"2024-04-02T16:11:32.179020+0000","last_became_active":"2024-04-02T16:11:32.171185+0000","last_became_peered":"2024-04-02T16:11:32.171185+0000","last_unstale":"2024-04-02T16:11:32.179020+0000","last_undegraded":"2024-04-02T16:11:32.179020+0000","last_fullsized":"2024-04-02T16:11:32.179020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:36.835221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161390+0000","last_change":"2024-04-02T16:11:32.174173+0000","last_active":"2024-04-02T16:11:34.161390+0000","last_peered":"2024-04-02T16:11:34.161390+0000","last_clean":"2024-04-02T16:11:34.161390+0000","last_became_active":"2024-04-02T16:11:32.173799+0000","last_became_peered":"2024-04-02T16:11:32.173799+0000","last_unstale":"2024-04-02T16:11:34.161390+0000","last_undegraded":"2024-04-02T16:11:34.161390+0000","last_fullsized":"2024-04-02T16:11:34.161390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156692+0000","last_change":"2024-04-02T16:11:32.169323+0000","last_active":"2024-04-02T16:11:33.156692+0000","last_peered":"2024-04-02T16:11:33.156692+0000","last_clean":"2024-04-02T16:11:33.156692+0000","last_became_active":"2024-04-02T16:11:32.169028+0000","last_became_peered":"2024-04-02T16:11:32.169028+0000","last_unstale":"2024-04-02T16:11:33.156692+0000","last_undegraded":"2024-04-02T16:11:33.156692+0000","last_fullsized":"2024-04-02T16:11:33.156692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179222+0000","last_change":"2024-04-02T16:11:32.177772+0000","last_active":"2024-04-02T16:11:32.179222+0000","last_peered":"2024-04-02T16:11:32.179222+0000","last_clean":"2024-04-02T16:11:32.179222+0000","last_became_active":"2024-04-02T16:11:32.177123+0000","last_became_peered":"2024-04-02T16:11:32.177123+0000","last_unstale":"2024-04-02T16:11:32.179222+0000","last_undegraded":"2024-04-02T16:11:32.179222+0000","last_fullsized":"2024-04-02T16:11:32.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:28:10.993274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.171094+0000","last_change":"2024-04-02T16:11:32.168117+0000","last_active":"2024-04-02T16:11:32.171094+0000","last_peered":"2024-04-02T16:11:32.171094+0000","last_clean":"2024-04-02T16:11:32.171094+0000","last_became_active":"2024-04-02T16:11:32.167687+0000","last_became_peered":"2024-04-02T16:11:32.167687+0000","last_unstale":"2024-04-02T16:11:32.171094+0000","last_undegraded":"2024-04-02T16:11:32.171094+0000","last_fullsized":"2024-04-02T16:11:32.171094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:36.835221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158588+0000","last_change":"2024-04-02T16:11:32.168501+0000","last_active":"2024-04-02T16:11:33.158588+0000","last_peered":"2024-04-02T16:11:33.158588+0000","last_clean":"2024-04-02T16:11:33.158588+0000","last_became_active":"2024-04-02T16:11:32.168164+0000","last_became_peered":"2024-04-02T16:11:32.168164+0000","last_unstale":"2024-04-02T16:11:33.158588+0000","last_undegraded":"2024-04-02T16:11:33.158588+0000","last_fullsized":"2024-04-02T16:11:33.158588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.157254+0000","last_change":"2024-04-02T16:11:32.152124+0000","last_active":"2024-04-02T16:11:32.157254+0000","last_peered":"2024-04-02T16:11:32.157254+0000","last_clean":"2024-04-02T16:11:32.157254+0000","last_became_active":"2024-04-02T16:11:32.150859+0000","last_became_peered":"2024-04-02T16:11:32.150859+0000","last_unstale":"2024-04-02T16:11:32.157254+0000","last_undegraded":"2024-04-02T16:11:32.157254+0000","last_fullsized":"2024-04-02T16:11:32.157254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160495+0000","last_change":"2024-04-02T16:11:32.166672+0000","last_active":"2024-04-02T16:11:34.160495+0000","last_peered":"2024-04-02T16:11:34.160495+0000","last_clean":"2024-04-02T16:11:34.160495+0000","last_became_active":"2024-04-02T16:11:32.165841+0000","last_became_peered":"2024-04-02T16:11:32.165841+0000","last_unstale":"2024-04-02T16:11:34.160495+0000","last_undegraded":"2024-04-02T16:11:34.160495+0000","last_fullsized":"2024-04-02T16:11:34.160495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157143+0000","last_change":"2024-04-02T16:11:32.148450+0000","last_active":"2024-04-02T16:11:33.157143+0000","last_peered":"2024-04-02T16:11:33.157143+0000","last_clean":"2024-04-02T16:11:33.157143+0000","last_became_active":"2024-04-02T16:11:32.147722+0000","last_became_peered":"2024-04-02T16:11:32.147722+0000","last_unstale":"2024-04-02T16:11:33.157143+0000","last_undegraded":"2024-04-02T16:11:33.157143+0000","last_fullsized":"2024-04-02T16:11:33.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.172261+0000","last_change":"2024-04-02T16:11:31.123742+0000","last_active":"2024-04-02T16:11:32.172261+0000","last_peered":"2024-04-02T16:11:32.172261+0000","last_clean":"2024-04-02T16:11:32.172261+0000","last_became_active":"2024-04-02T16:11:31.123404+0000","last_became_peered":"2024-04-02T16:11:31.123404+0000","last_unstale":"2024-04-02T16:11:32.172261+0000","last_undegraded":"2024-04-02T16:11:32.172261+0000","last_fullsized":"2024-04-02T16:11:32.172261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158329+0000","last_change":"2024-04-02T16:11:32.140773+0000","last_active":"2024-04-02T16:11:32.158329+0000","last_peered":"2024-04-02T16:11:32.158329+0000","last_clean":"2024-04-02T16:11:32.158329+0000","last_became_active":"2024-04-02T16:11:32.140008+0000","last_became_peered":"2024-04-02T16:11:32.140008+0000","last_unstale":"2024-04-02T16:11:32.158329+0000","last_undegraded":"2024-04-02T16:11:32.158329+0000","last_fullsized":"2024-04-02T16:11:32.158329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:13.477320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160916+0000","last_change":"2024-04-02T16:11:31.131833+0000","last_active":"2024-04-02T16:11:34.160916+0000","last_peered":"2024-04-02T16:11:34.160916+0000","last_clean":"2024-04-02T16:11:34.160916+0000","last_became_active":"2024-04-02T16:11:31.130705+0000","last_became_peered":"2024-04-02T16:11:31.130705+0000","last_unstale":"2024-04-02T16:11:34.160916+0000","last_undegraded":"2024-04-02T16:11:34.160916+0000","last_fullsized":"2024-04-02T16:11:34.160916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.150813+0000","last_change":"2024-04-02T16:11:31.130860+0000","last_active":"2024-04-02T16:11:32.150813+0000","last_peered":"2024-04-02T16:11:32.150813+0000","last_clean":"2024-04-02T16:11:32.150813+0000","last_became_active":"2024-04-02T16:11:31.130391+0000","last_became_peered":"2024-04-02T16:11:31.130391+0000","last_unstale":"2024-04-02T16:11:32.150813+0000","last_undegraded":"2024-04-02T16:11:32.150813+0000","last_fullsized":"2024-04-02T16:11:32.150813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178184+0000","last_change":"2024-04-02T16:11:32.166977+0000","last_active":"2024-04-02T16:11:32.178184+0000","last_peered":"2024-04-02T16:11:32.178184+0000","last_clean":"2024-04-02T16:11:32.178184+0000","last_became_active":"2024-04-02T16:11:32.166690+0000","last_became_peered":"2024-04-02T16:11:32.166690+0000","last_unstale":"2024-04-02T16:11:32.178184+0000","last_undegraded":"2024-04-02T16:11:32.178184+0000","last_fullsized":"2024-04-02T16:11:32.178184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157628+0000","last_change":"2024-04-02T16:11:31.128418+0000","last_active":"2024-04-02T16:11:33.157628+0000","last_peered":"2024-04-02T16:11:33.157628+0000","last_clean":"2024-04-02T16:11:33.157628+0000","last_became_active":"2024-04-02T16:11:31.126329+0000","last_became_peered":"2024-04-02T16:11:31.126329+0000","last_unstale":"2024-04-02T16:11:33.157628+0000","last_undegraded":"2024-04-02T16:11:33.157628+0000","last_fullsized":"2024-04-02T16:11:33.157628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156477+0000","last_change":"2024-04-02T16:11:32.164197+0000","last_active":"2024-04-02T16:11:33.156477+0000","last_peered":"2024-04-02T16:11:33.156477+0000","last_clean":"2024-04-02T16:11:33.156477+0000","last_became_active":"2024-04-02T16:11:32.163440+0000","last_became_peered":"2024-04-02T16:11:32.163440+0000","last_unstale":"2024-04-02T16:11:33.156477+0000","last_undegraded":"2024-04-02T16:11:33.156477+0000","last_fullsized":"2024-04-02T16:11:33.156477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161113+0000","last_change":"2024-04-02T16:11:31.137185+0000","last_active":"2024-04-02T16:11:34.161113+0000","last_peered":"2024-04-02T16:11:34.161113+0000","last_clean":"2024-04-02T16:11:34.161113+0000","last_became_active":"2024-04-02T16:11:31.136386+0000","last_became_peered":"2024-04-02T16:11:31.136386+0000","last_unstale":"2024-04-02T16:11:34.161113+0000","last_undegraded":"2024-04-02T16:11:34.161113+0000","last_fullsized":"2024-04-02T16:11:34.161113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158908+0000","last_change":"2024-04-02T16:11:32.159269+0000","last_active":"2024-04-02T16:11:33.158908+0000","last_peered":"2024-04-02T16:11:33.158908+0000","last_clean":"2024-04-02T16:11:33.158908+0000","last_became_active":"2024-04-02T16:11:32.158906+0000","last_became_peered":"2024-04-02T16:11:32.158906+0000","last_unstale":"2024-04-02T16:11:33.158908+0000","last_undegraded":"2024-04-02T16:11:33.158908+0000","last_fullsized":"2024-04-02T16:11:33.158908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159157+0000","last_change":"2024-04-02T16:11:31.133206+0000","last_active":"2024-04-02T16:11:33.159157+0000","last_peered":"2024-04-02T16:11:33.159157+0000","last_clean":"2024-04-02T16:11:33.159157+0000","last_became_active":"2024-04-02T16:11:31.132012+0000","last_became_peered":"2024-04-02T16:11:31.132012+0000","last_unstale":"2024-04-02T16:11:33.159157+0000","last_undegraded":"2024-04-02T16:11:33.159157+0000","last_fullsized":"2024-04-02T16:11:33.159157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158578+0000","last_change":"2024-04-02T16:11:32.153558+0000","last_active":"2024-04-02T16:11:32.158578+0000","last_peered":"2024-04-02T16:11:32.158578+0000","last_clean":"2024-04-02T16:11:32.158578+0000","last_became_active":"2024-04-02T16:11:32.152697+0000","last_became_peered":"2024-04-02T16:11:32.152697+0000","last_unstale":"2024-04-02T16:11:32.158578+0000","last_undegraded":"2024-04-02T16:11:32.158578+0000","last_fullsized":"2024-04-02T16:11:32.158578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156779+0000","last_change":"2024-04-02T16:11:31.134656+0000","last_active":"2024-04-02T16:11:33.156779+0000","last_peered":"2024-04-02T16:11:33.156779+0000","last_clean":"2024-04-02T16:11:33.156779+0000","last_became_active":"2024-04-02T16:11:31.134191+0000","last_became_peered":"2024-04-02T16:11:31.134191+0000","last_unstale":"2024-04-02T16:11:33.156779+0000","last_undegraded":"2024-04-02T16:11:33.156779+0000","last_fullsized":"2024-04-02T16:11:33.156779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158486+0000","last_change":"2024-04-02T16:11:32.154312+0000","last_active":"2024-04-02T16:11:33.158486+0000","last_peered":"2024-04-02T16:11:33.158486+0000","last_clean":"2024-04-02T16:11:33.158486+0000","last_became_active":"2024-04-02T16:11:32.153984+0000","last_became_peered":"2024-04-02T16:11:32.153984+0000","last_unstale":"2024-04-02T16:11:33.158486+0000","last_undegraded":"2024-04-02T16:11:33.158486+0000","last_fullsized":"2024-04-02T16:11:33.158486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161022+0000","last_change":"2024-04-02T16:11:31.128529+0000","last_active":"2024-04-02T16:11:34.161022+0000","last_peered":"2024-04-02T16:11:34.161022+0000","last_clean":"2024-04-02T16:11:34.161022+0000","last_became_active":"2024-04-02T16:11:31.127760+0000","last_became_peered":"2024-04-02T16:11:31.127760+0000","last_unstale":"2024-04-02T16:11:34.161022+0000","last_undegraded":"2024-04-02T16:11:34.161022+0000","last_fullsized":"2024-04-02T16:11:34.161022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497089+0000","last_change":"2024-04-02T16:11:32.169840+0000","last_active":"2024-04-02T16:11:33.497089+0000","last_peered":"2024-04-02T16:11:33.497089+0000","last_clean":"2024-04-02T16:11:33.497089+0000","last_became_active":"2024-04-02T16:11:32.167352+0000","last_became_peered":"2024-04-02T16:11:32.167352+0000","last_unstale":"2024-04-02T16:11:33.497089+0000","last_undegraded":"2024-04-02T16:11:33.497089+0000","last_fullsized":"2024-04-02T16:11:33.497089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.174627+0000","last_change":"2024-04-02T16:11:31.135077+0000","last_active":"2024-04-02T16:11:32.174627+0000","last_peered":"2024-04-02T16:11:32.174627+0000","last_clean":"2024-04-02T16:11:32.174627+0000","last_became_active":"2024-04-02T16:11:31.134519+0000","last_became_peered":"2024-04-02T16:11:31.134519+0000","last_unstale":"2024-04-02T16:11:32.174627+0000","last_undegraded":"2024-04-02T16:11:32.174627+0000","last_fullsized":"2024-04-02T16:11:32.174627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.169908+0000","last_change":"2024-04-02T16:11:32.155398+0000","last_active":"2024-04-02T16:11:32.169908+0000","last_peered":"2024-04-02T16:11:32.169908+0000","last_clean":"2024-04-02T16:11:32.169908+0000","last_became_active":"2024-04-02T16:11:32.154518+0000","last_became_peered":"2024-04-02T16:11:32.154518+0000","last_unstale":"2024-04-02T16:11:32.169908+0000","last_undegraded":"2024-04-02T16:11:32.169908+0000","last_fullsized":"2024-04-02T16:11:32.169908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:55.127710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160202+0000","last_change":"2024-04-02T16:11:31.125926+0000","last_active":"2024-04-02T16:11:33.160202+0000","last_peered":"2024-04-02T16:11:33.160202+0000","last_clean":"2024-04-02T16:11:33.160202+0000","last_became_active":"2024-04-02T16:11:31.124109+0000","last_became_peered":"2024-04-02T16:11:31.124109+0000","last_unstale":"2024-04-02T16:11:33.160202+0000","last_undegraded":"2024-04-02T16:11:33.160202+0000","last_fullsized":"2024-04-02T16:11:33.160202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497615+0000","last_change":"2024-04-02T16:11:32.171153+0000","last_active":"2024-04-02T16:11:33.497615+0000","last_peered":"2024-04-02T16:11:33.497615+0000","last_clean":"2024-04-02T16:11:33.497615+0000","last_became_active":"2024-04-02T16:11:32.170399+0000","last_became_peered":"2024-04-02T16:11:32.170399+0000","last_unstale":"2024-04-02T16:11:33.497615+0000","last_undegraded":"2024-04-02T16:11:33.497615+0000","last_fullsized":"2024-04-02T16:11:33.497615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160399+0000","last_change":"2024-04-02T16:11:31.131680+0000","last_active":"2024-04-02T16:11:34.160399+0000","last_peered":"2024-04-02T16:11:34.160399+0000","last_clean":"2024-04-02T16:11:34.160399+0000","last_became_active":"2024-04-02T16:11:31.129432+0000","last_became_peered":"2024-04-02T16:11:31.129432+0000","last_unstale":"2024-04-02T16:11:34.160399+0000","last_undegraded":"2024-04-02T16:11:34.160399+0000","last_fullsized":"2024-04-02T16:11:34.160399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160538+0000","last_change":"2024-04-02T16:11:32.167516+0000","last_active":"2024-04-02T16:11:34.160538+0000","last_peered":"2024-04-02T16:11:34.160538+0000","last_clean":"2024-04-02T16:11:34.160538+0000","last_became_active":"2024-04-02T16:11:32.167201+0000","last_became_peered":"2024-04-02T16:11:32.167201+0000","last_unstale":"2024-04-02T16:11:34.160538+0000","last_undegraded":"2024-04-02T16:11:34.160538+0000","last_fullsized":"2024-04-02T16:11:34.160538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.175668+0000","last_change":"2024-04-02T16:11:31.117868+0000","last_active":"2024-04-02T16:11:32.175668+0000","last_peered":"2024-04-02T16:11:32.175668+0000","last_clean":"2024-04-02T16:11:32.175668+0000","last_became_active":"2024-04-02T16:11:31.117437+0000","last_became_peered":"2024-04-02T16:11:31.117437+0000","last_unstale":"2024-04-02T16:11:32.175668+0000","last_undegraded":"2024-04-02T16:11:32.175668+0000","last_fullsized":"2024-04-02T16:11:32.175668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497088+0000","last_change":"2024-04-02T16:11:32.178110+0000","last_active":"2024-04-02T16:11:33.497088+0000","last_peered":"2024-04-02T16:11:33.497088+0000","last_clean":"2024-04-02T16:11:33.497088+0000","last_became_active":"2024-04-02T16:11:32.177756+0000","last_became_peered":"2024-04-02T16:11:32.177756+0000","last_unstale":"2024-04-02T16:11:33.497088+0000","last_undegraded":"2024-04-02T16:11:33.497088+0000","last_fullsized":"2024-04-02T16:11:33.497088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159238+0000","last_change":"2024-04-02T16:11:31.130795+0000","last_active":"2024-04-02T16:11:33.159238+0000","last_peered":"2024-04-02T16:11:33.159238+0000","last_clean":"2024-04-02T16:11:33.159238+0000","last_became_active":"2024-04-02T16:11:31.129907+0000","last_became_peered":"2024-04-02T16:11:31.129907+0000","last_unstale":"2024-04-02T16:11:33.159238+0000","last_undegraded":"2024-04-02T16:11:33.159238+0000","last_fullsized":"2024-04-02T16:11:33.159238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:27:40.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497755+0000","last_change":"2024-04-02T16:11:32.173519+0000","last_active":"2024-04-02T16:11:33.497755+0000","last_peered":"2024-04-02T16:11:33.497755+0000","last_clean":"2024-04-02T16:11:33.497755+0000","last_became_active":"2024-04-02T16:11:32.173159+0000","last_became_peered":"2024-04-02T16:11:32.173159+0000","last_unstale":"2024-04-02T16:11:33.497755+0000","last_undegraded":"2024-04-02T16:11:33.497755+0000","last_fullsized":"2024-04-02T16:11:33.497755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178348+0000","last_change":"2024-04-02T16:11:31.113477+0000","last_active":"2024-04-02T16:11:32.178348+0000","last_peered":"2024-04-02T16:11:32.178348+0000","last_clean":"2024-04-02T16:11:32.178348+0000","last_became_active":"2024-04-02T16:11:31.112346+0000","last_became_peered":"2024-04-02T16:11:31.112346+0000","last_unstale":"2024-04-02T16:11:32.178348+0000","last_undegraded":"2024-04-02T16:11:32.178348+0000","last_fullsized":"2024-04-02T16:11:32.178348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498157+0000","last_change":"2024-04-02T16:11:32.173602+0000","last_active":"2024-04-02T16:11:33.498157+0000","last_peered":"2024-04-02T16:11:33.498157+0000","last_clean":"2024-04-02T16:11:33.498157+0000","last_became_active":"2024-04-02T16:11:32.173294+0000","last_became_peered":"2024-04-02T16:11:32.173294+0000","last_unstale":"2024-04-02T16:11:33.498157+0000","last_undegraded":"2024-04-02T16:11:33.498157+0000","last_fullsized":"2024-04-02T16:11:33.498157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:50.854339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156803+0000","last_change":"2024-04-02T16:11:31.134514+0000","last_active":"2024-04-02T16:11:33.156803+0000","last_peered":"2024-04-02T16:11:33.156803+0000","last_clean":"2024-04-02T16:11:33.156803+0000","last_became_active":"2024-04-02T16:11:31.134123+0000","last_became_peered":"2024-04-02T16:11:31.134123+0000","last_unstale":"2024-04-02T16:11:33.156803+0000","last_undegraded":"2024-04-02T16:11:33.156803+0000","last_fullsized":"2024-04-02T16:11:33.156803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159396+0000","last_change":"2024-04-02T16:11:32.172101+0000","last_active":"2024-04-02T16:11:33.159396+0000","last_peered":"2024-04-02T16:11:33.159396+0000","last_clean":"2024-04-02T16:11:33.159396+0000","last_became_active":"2024-04-02T16:11:32.169192+0000","last_became_peered":"2024-04-02T16:11:32.169192+0000","last_unstale":"2024-04-02T16:11:33.159396+0000","last_undegraded":"2024-04-02T16:11:33.159396+0000","last_fullsized":"2024-04-02T16:11:33.159396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.133922+0000","last_change":"2024-04-02T16:11:31.118803+0000","last_active":"2024-04-02T16:11:32.133922+0000","last_peered":"2024-04-02T16:11:32.133922+0000","last_clean":"2024-04-02T16:11:32.133922+0000","last_became_active":"2024-04-02T16:11:31.118400+0000","last_became_peered":"2024-04-02T16:11:31.118400+0000","last_unstale":"2024-04-02T16:11:32.133922+0000","last_undegraded":"2024-04-02T16:11:32.133922+0000","last_fullsized":"2024-04-02T16:11:32.133922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497571+0000","last_change":"2024-04-02T16:11:32.175909+0000","last_active":"2024-04-02T16:11:33.497571+0000","last_peered":"2024-04-02T16:11:33.497571+0000","last_clean":"2024-04-02T16:11:33.497571+0000","last_became_active":"2024-04-02T16:11:32.175629+0000","last_became_peered":"2024-04-02T16:11:32.175629+0000","last_unstale":"2024-04-02T16:11:33.497571+0000","last_undegraded":"2024-04-02T16:11:33.497571+0000","last_fullsized":"2024-04-02T16:11:33.497571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496798+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:11:33.496798+0000","last_peered":"2024-04-02T16:11:33.496798+0000","last_clean":"2024-04-02T16:11:33.496798+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:11:33.496798+0000","last_undegraded":"2024-04-02T16:11:33.496798+0000","last_fullsized":"2024-04-02T16:11:33.496798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158329+0000","last_change":"2024-04-02T16:11:31.114855+0000","last_active":"2024-04-02T16:11:33.158329+0000","last_peered":"2024-04-02T16:11:33.158329+0000","last_clean":"2024-04-02T16:11:33.158329+0000","last_became_active":"2024-04-02T16:11:31.114136+0000","last_became_peered":"2024-04-02T16:11:31.114136+0000","last_unstale":"2024-04-02T16:11:33.158329+0000","last_undegraded":"2024-04-02T16:11:33.158329+0000","last_fullsized":"2024-04-02T16:11:33.158329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.179205+0000","last_change":"2024-04-02T16:11:32.177918+0000","last_active":"2024-04-02T16:11:32.179205+0000","last_peered":"2024-04-02T16:11:32.179205+0000","last_clean":"2024-04-02T16:11:32.179205+0000","last_became_active":"2024-04-02T16:11:32.177523+0000","last_became_peered":"2024-04-02T16:11:32.177523+0000","last_unstale":"2024-04-02T16:11:32.179205+0000","last_undegraded":"2024-04-02T16:11:32.179205+0000","last_fullsized":"2024-04-02T16:11:32.179205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:27.528611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496680+0000","last_change":"2024-04-02T16:11:31.147127+0000","last_active":"2024-04-02T16:11:33.496680+0000","last_peered":"2024-04-02T16:11:33.496680+0000","last_clean":"2024-04-02T16:11:33.496680+0000","last_became_active":"2024-04-02T16:11:31.146323+0000","last_became_peered":"2024-04-02T16:11:31.146323+0000","last_unstale":"2024-04-02T16:11:33.496680+0000","last_undegraded":"2024-04-02T16:11:33.496680+0000","last_fullsized":"2024-04-02T16:11:33.496680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158507+0000","last_change":"2024-04-02T16:11:32.152084+0000","last_active":"2024-04-02T16:11:32.158507+0000","last_peered":"2024-04-02T16:11:32.158507+0000","last_clean":"2024-04-02T16:11:32.158507+0000","last_became_active":"2024-04-02T16:11:32.151565+0000","last_became_peered":"2024-04-02T16:11:32.151565+0000","last_unstale":"2024-04-02T16:11:32.158507+0000","last_undegraded":"2024-04-02T16:11:32.158507+0000","last_fullsized":"2024-04-02T16:11:32.158507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.134373+0000","last_change":"2024-04-02T16:11:31.112830+0000","last_active":"2024-04-02T16:11:32.134373+0000","last_peered":"2024-04-02T16:11:32.134373+0000","last_clean":"2024-04-02T16:11:32.134373+0000","last_became_active":"2024-04-02T16:11:31.112472+0000","last_became_peered":"2024-04-02T16:11:31.112472+0000","last_unstale":"2024-04-02T16:11:32.134373+0000","last_undegraded":"2024-04-02T16:11:32.134373+0000","last_fullsized":"2024-04-02T16:11:32.134373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156504+0000","last_change":"2024-04-02T16:11:32.152287+0000","last_active":"2024-04-02T16:11:33.156504+0000","last_peered":"2024-04-02T16:11:33.156504+0000","last_clean":"2024-04-02T16:11:33.156504+0000","last_became_active":"2024-04-02T16:11:32.151559+0000","last_became_peered":"2024-04-02T16:11:32.151559+0000","last_unstale":"2024-04-02T16:11:33.156504+0000","last_undegraded":"2024-04-02T16:11:33.156504+0000","last_fullsized":"2024-04-02T16:11:33.156504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158997+0000","last_change":"2024-04-02T16:11:31.131581+0000","last_active":"2024-04-02T16:11:33.158997+0000","last_peered":"2024-04-02T16:11:33.158997+0000","last_clean":"2024-04-02T16:11:33.158997+0000","last_became_active":"2024-04-02T16:11:31.130730+0000","last_became_peered":"2024-04-02T16:11:31.130730+0000","last_unstale":"2024-04-02T16:11:33.158997+0000","last_undegraded":"2024-04-02T16:11:33.158997+0000","last_fullsized":"2024-04-02T16:11:33.158997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157459+0000","last_change":"2024-04-02T16:11:32.159434+0000","last_active":"2024-04-02T16:11:33.157459+0000","last_peered":"2024-04-02T16:11:33.157459+0000","last_clean":"2024-04-02T16:11:33.157459+0000","last_became_active":"2024-04-02T16:11:32.159056+0000","last_became_peered":"2024-04-02T16:11:32.159056+0000","last_unstale":"2024-04-02T16:11:33.157459+0000","last_undegraded":"2024-04-02T16:11:33.157459+0000","last_fullsized":"2024-04-02T16:11:33.157459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156640+0000","last_change":"2024-04-02T16:11:31.131364+0000","last_active":"2024-04-02T16:11:33.156640+0000","last_peered":"2024-04-02T16:11:33.156640+0000","last_clean":"2024-04-02T16:11:33.156640+0000","last_became_active":"2024-04-02T16:11:31.130814+0000","last_became_peered":"2024-04-02T16:11:31.130814+0000","last_unstale":"2024-04-02T16:11:33.156640+0000","last_undegraded":"2024-04-02T16:11:33.156640+0000","last_fullsized":"2024-04-02T16:11:33.156640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160072+0000","last_change":"2024-04-02T16:11:32.168617+0000","last_active":"2024-04-02T16:11:33.160072+0000","last_peered":"2024-04-02T16:11:33.160072+0000","last_clean":"2024-04-02T16:11:33.160072+0000","last_became_active":"2024-04-02T16:11:32.168318+0000","last_became_peered":"2024-04-02T16:11:32.168318+0000","last_unstale":"2024-04-02T16:11:33.160072+0000","last_undegraded":"2024-04-02T16:11:33.160072+0000","last_fullsized":"2024-04-02T16:11:33.160072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497910+0000","last_change":"2024-04-02T16:11:31.150504+0000","last_active":"2024-04-02T16:11:33.497910+0000","last_peered":"2024-04-02T16:11:33.497910+0000","last_clean":"2024-04-02T16:11:33.497910+0000","last_became_active":"2024-04-02T16:11:31.149962+0000","last_became_peered":"2024-04-02T16:11:31.149962+0000","last_unstale":"2024-04-02T16:11:33.497910+0000","last_undegraded":"2024-04-02T16:11:33.497910+0000","last_fullsized":"2024-04-02T16:11:33.497910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:35.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156819+0000","last_change":"2024-04-02T16:11:32.157220+0000","last_active":"2024-04-02T16:11:33.156819+0000","last_peered":"2024-04-02T16:11:33.156819+0000","last_clean":"2024-04-02T16:11:33.156819+0000","last_became_active":"2024-04-02T16:11:32.156725+0000","last_became_peered":"2024-04-02T16:11:32.156725+0000","last_unstale":"2024-04-02T16:11:33.156819+0000","last_undegraded":"2024-04-02T16:11:33.156819+0000","last_fullsized":"2024-04-02T16:11:33.156819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.159254+0000","last_change":"2024-04-02T16:11:31.130016+0000","last_active":"2024-04-02T16:11:34.159254+0000","last_peered":"2024-04-02T16:11:34.159254+0000","last_clean":"2024-04-02T16:11:34.159254+0000","last_became_active":"2024-04-02T16:11:31.129578+0000","last_became_peered":"2024-04-02T16:11:31.129578+0000","last_unstale":"2024-04-02T16:11:34.159254+0000","last_undegraded":"2024-04-02T16:11:34.159254+0000","last_fullsized":"2024-04-02T16:11:34.159254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.165198+0000","last_change":"2024-04-02T16:11:32.162159+0000","last_active":"2024-04-02T16:11:32.165198+0000","last_peered":"2024-04-02T16:11:32.165198+0000","last_clean":"2024-04-02T16:11:32.165198+0000","last_became_active":"2024-04-02T16:11:32.161875+0000","last_became_peered":"2024-04-02T16:11:32.161875+0000","last_unstale":"2024-04-02T16:11:32.165198+0000","last_undegraded":"2024-04-02T16:11:32.165198+0000","last_fullsized":"2024-04-02T16:11:32.165198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161008+0000","last_change":"2024-04-02T16:11:31.143750+0000","last_active":"2024-04-02T16:11:34.161008+0000","last_peered":"2024-04-02T16:11:34.161008+0000","last_clean":"2024-04-02T16:11:34.161008+0000","last_became_active":"2024-04-02T16:11:31.142991+0000","last_became_peered":"2024-04-02T16:11:31.142991+0000","last_unstale":"2024-04-02T16:11:34.161008+0000","last_undegraded":"2024-04-02T16:11:34.161008+0000","last_fullsized":"2024-04-02T16:11:34.161008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.161375+0000","last_change":"2024-04-02T16:11:32.153539+0000","last_active":"2024-04-02T16:11:32.161375+0000","last_peered":"2024-04-02T16:11:32.161375+0000","last_clean":"2024-04-02T16:11:32.161375+0000","last_became_active":"2024-04-02T16:11:32.152769+0000","last_became_peered":"2024-04-02T16:11:32.152769+0000","last_unstale":"2024-04-02T16:11:32.161375+0000","last_undegraded":"2024-04-02T16:11:32.161375+0000","last_fullsized":"2024-04-02T16:11:32.161375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496646+0000","last_change":"2024-04-02T16:11:31.144365+0000","last_active":"2024-04-02T16:11:33.496646+0000","last_peered":"2024-04-02T16:11:33.496646+0000","last_clean":"2024-04-02T16:11:33.496646+0000","last_became_active":"2024-04-02T16:11:31.143659+0000","last_became_peered":"2024-04-02T16:11:31.143659+0000","last_unstale":"2024-04-02T16:11:33.496646+0000","last_undegraded":"2024-04-02T16:11:33.496646+0000","last_fullsized":"2024-04-02T16:11:33.496646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157111+0000","last_change":"2024-04-02T16:11:32.153561+0000","last_active":"2024-04-02T16:11:33.157111+0000","last_peered":"2024-04-02T16:11:33.157111+0000","last_clean":"2024-04-02T16:11:33.157111+0000","last_became_active":"2024-04-02T16:11:32.148826+0000","last_became_peered":"2024-04-02T16:11:32.148826+0000","last_unstale":"2024-04-02T16:11:33.157111+0000","last_undegraded":"2024-04-02T16:11:33.157111+0000","last_fullsized":"2024-04-02T16:11:33.157111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156589+0000","last_change":"2024-04-02T16:11:31.144769+0000","last_active":"2024-04-02T16:11:33.156589+0000","last_peered":"2024-04-02T16:11:33.156589+0000","last_clean":"2024-04-02T16:11:33.156589+0000","last_became_active":"2024-04-02T16:11:31.144386+0000","last_became_peered":"2024-04-02T16:11:31.144386+0000","last_unstale":"2024-04-02T16:11:33.156589+0000","last_undegraded":"2024-04-02T16:11:33.156589+0000","last_fullsized":"2024-04-02T16:11:33.156589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.164266+0000","last_change":"2024-04-02T16:11:32.160650+0000","last_active":"2024-04-02T16:11:32.164266+0000","last_peered":"2024-04-02T16:11:32.164266+0000","last_clean":"2024-04-02T16:11:32.164266+0000","last_became_active":"2024-04-02T16:11:32.160239+0000","last_became_peered":"2024-04-02T16:11:32.160239+0000","last_unstale":"2024-04-02T16:11:32.164266+0000","last_undegraded":"2024-04-02T16:11:32.164266+0000","last_fullsized":"2024-04-02T16:11:32.164266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:23.736628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157210+0000","last_change":"2024-04-02T16:11:31.119214+0000","last_active":"2024-04-02T16:11:33.157210+0000","last_peered":"2024-04-02T16:11:33.157210+0000","last_clean":"2024-04-02T16:11:33.157210+0000","last_became_active":"2024-04-02T16:11:31.118885+0000","last_became_peered":"2024-04-02T16:11:31.118885+0000","last_unstale":"2024-04-02T16:11:33.157210+0000","last_undegraded":"2024-04-02T16:11:33.157210+0000","last_fullsized":"2024-04-02T16:11:33.157210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.177821+0000","last_change":"2024-04-02T16:11:32.165991+0000","last_active":"2024-04-02T16:11:32.177821+0000","last_peered":"2024-04-02T16:11:32.177821+0000","last_clean":"2024-04-02T16:11:32.177821+0000","last_became_active":"2024-04-02T16:11:32.165617+0000","last_became_peered":"2024-04-02T16:11:32.165617+0000","last_unstale":"2024-04-02T16:11:32.177821+0000","last_undegraded":"2024-04-02T16:11:32.177821+0000","last_fullsized":"2024-04-02T16:11:32.177821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497809+0000","last_change":"2024-04-02T16:11:31.150660+0000","last_active":"2024-04-02T16:11:33.497809+0000","last_peered":"2024-04-02T16:11:33.497809+0000","last_clean":"2024-04-02T16:11:33.497809+0000","last_became_active":"2024-04-02T16:11:31.149922+0000","last_became_peered":"2024-04-02T16:11:31.149922+0000","last_unstale":"2024-04-02T16:11:33.497809+0000","last_undegraded":"2024-04-02T16:11:33.497809+0000","last_fullsized":"2024-04-02T16:11:33.497809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:54.118266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160589+0000","last_change":"2024-04-02T16:11:32.166809+0000","last_active":"2024-04-02T16:11:34.160589+0000","last_peered":"2024-04-02T16:11:34.160589+0000","last_clean":"2024-04-02T16:11:34.160589+0000","last_became_active":"2024-04-02T16:11:32.164926+0000","last_became_peered":"2024-04-02T16:11:32.164926+0000","last_unstale":"2024-04-02T16:11:34.160589+0000","last_undegraded":"2024-04-02T16:11:34.160589+0000","last_fullsized":"2024-04-02T16:11:34.160589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159873+0000","last_change":"2024-04-02T16:11:32.146387+0000","last_active":"2024-04-02T16:11:33.159873+0000","last_peered":"2024-04-02T16:11:33.159873+0000","last_clean":"2024-04-02T16:11:33.159873+0000","last_became_active":"2024-04-02T16:11:32.145655+0000","last_became_peered":"2024-04-02T16:11:32.145655+0000","last_unstale":"2024-04-02T16:11:33.159873+0000","last_undegraded":"2024-04-02T16:11:33.159873+0000","last_fullsized":"2024-04-02T16:11:33.159873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497313+0000","last_change":"2024-04-02T16:11:31.147426+0000","last_active":"2024-04-02T16:11:33.497313+0000","last_peered":"2024-04-02T16:11:33.497313+0000","last_clean":"2024-04-02T16:11:33.497313+0000","last_became_active":"2024-04-02T16:11:31.146661+0000","last_became_peered":"2024-04-02T16:11:31.146661+0000","last_unstale":"2024-04-02T16:11:33.497313+0000","last_undegraded":"2024-04-02T16:11:33.497313+0000","last_fullsized":"2024-04-02T16:11:33.497313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159146+0000","last_change":"2024-04-02T16:11:32.163890+0000","last_active":"2024-04-02T16:11:33.159146+0000","last_peered":"2024-04-02T16:11:33.159146+0000","last_clean":"2024-04-02T16:11:33.159146+0000","last_became_active":"2024-04-02T16:11:32.163585+0000","last_became_peered":"2024-04-02T16:11:32.163585+0000","last_unstale":"2024-04-02T16:11:33.159146+0000","last_undegraded":"2024-04-02T16:11:33.159146+0000","last_fullsized":"2024-04-02T16:11:33.159146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497629+0000","last_change":"2024-04-02T16:11:31.151627+0000","last_active":"2024-04-02T16:11:33.497629+0000","last_peered":"2024-04-02T16:11:33.497629+0000","last_clean":"2024-04-02T16:11:33.497629+0000","last_became_active":"2024-04-02T16:11:31.151077+0000","last_became_peered":"2024-04-02T16:11:31.151077+0000","last_unstale":"2024-04-02T16:11:33.497629+0000","last_undegraded":"2024-04-02T16:11:33.497629+0000","last_fullsized":"2024-04-02T16:11:33.497629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:47.686560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.158552+0000","last_change":"2024-04-02T16:11:32.153198+0000","last_active":"2024-04-02T16:11:32.158552+0000","last_peered":"2024-04-02T16:11:32.158552+0000","last_clean":"2024-04-02T16:11:32.158552+0000","last_became_active":"2024-04-02T16:11:32.149403+0000","last_became_peered":"2024-04-02T16:11:32.149403+0000","last_unstale":"2024-04-02T16:11:32.158552+0000","last_undegraded":"2024-04-02T16:11:32.158552+0000","last_fullsized":"2024-04-02T16:11:32.158552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178419+0000","last_change":"2024-04-02T16:11:31.128936+0000","last_active":"2024-04-02T16:11:32.178419+0000","last_peered":"2024-04-02T16:11:32.178419+0000","last_clean":"2024-04-02T16:11:32.178419+0000","last_became_active":"2024-04-02T16:11:31.128574+0000","last_became_peered":"2024-04-02T16:11:31.128574+0000","last_unstale":"2024-04-02T16:11:32.178419+0000","last_undegraded":"2024-04-02T16:11:32.178419+0000","last_fullsized":"2024-04-02T16:11:32.178419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.178613+0000","last_change":"2024-04-02T16:11:32.170441+0000","last_active":"2024-04-02T16:11:32.178613+0000","last_peered":"2024-04-02T16:11:32.178613+0000","last_clean":"2024-04-02T16:11:32.178613+0000","last_became_active":"2024-04-02T16:11:32.167070+0000","last_became_peered":"2024-04-02T16:11:32.167070+0000","last_unstale":"2024-04-02T16:11:32.178613+0000","last_undegraded":"2024-04-02T16:11:32.178613+0000","last_fullsized":"2024-04-02T16:11:32.178613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-02T16:11:32.174093+0000","last_change":"2024-04-02T16:11:31.142814+0000","last_active":"2024-04-02T16:11:32.174093+0000","last_peered":"2024-04-02T16:11:32.174093+0000","last_clean":"2024-04-02T16:11:32.174093+0000","last_became_active":"2024-04-02T16:11:31.142568+0000","last_became_peered":"2024-04-02T16:11:31.142568+0000","last_unstale":"2024-04-02T16:11:32.174093+0000","last_undegraded":"2024-04-02T16:11:32.174093+0000","last_fullsized":"2024-04-02T16:11:32.174093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:11:36.580 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T16:11:36.580 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T16:11:36.580 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-02T16:11:36.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-02T16:11:36.581 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-02T16:11:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 Processor -- start 2024-04-02T16:11:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 -- start start 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab4141c30 0x7f1ab414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 0x7f1ab4152110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab414baa0 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab40b6120 con 0x7f1ab4141c30 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1abbbe9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab40bdd80 con 0x7f1ab414b630 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab4141c30 0x7f1ab414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab4141c30 0x7f1ab414b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37036/0 (socket says 172.21.15.40:37036) 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 -- 172.21.15.40:0/2267969657 learned_addr learned my addr 172.21.15.40:0/2267969657 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1aba186640 1 --2- 172.21.15.40:0/2267969657 >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 0x7f1ab4152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 -- 172.21.15.40:0/2267969657 >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 msgr2=0x7f1ab4152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 --2- 172.21.15.40:0/2267969657 >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 0x7f1ab4152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 -- 172.21.15.40:0/2267969657 >> v1:172.21.15.204:6789/0 conn(0x7f1ab40bd9a0 legacy=0x7f1ab4141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1ab9184640 1 -- 172.21.15.40:0/2267969657 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40b5b60 con 0x7f1ab4141c30 2024-04-02T16:11:36.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.733+0000 7f1aba186640 1 --2- 172.21.15.40:0/2267969657 >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 0x7f1ab4152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab9184640 1 --2- 172.21.15.40:0/2267969657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab4141c30 0x7f1ab414b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406e5e0 tx=0x7f1aa401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6883b6599c977f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab8983640 1 -- 172.21.15.40:0/2267969657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40b0070 con 0x7f1ab4141c30 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab8983640 1 -- 172.21.15.40:0/2267969657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa40ab600 con 0x7f1ab4141c30 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab8983640 1 -- 172.21.15.40:0/2267969657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40ab9d0 con 0x7f1ab4141c30 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- 172.21.15.40:0/2267969657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab4141c30 msgr2=0x7f1ab414b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/2267969657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab4141c30 0x7f1ab414b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406e5e0 tx=0x7f1aa401c400 comp rx=0 tx=0).stop 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- 172.21.15.40:0/2267969657 shutdown_connections 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/2267969657 >> v2:172.21.15.204:3300/0 conn(0x7f1ab414b630 0x7f1ab4152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/2267969657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab4141c30 0x7f1ab414b0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- 172.21.15.40:0/2267969657 >> 172.21.15.40:0/2267969657 conn(0x7f1ab4073310 msgr2=0x7f1ab40bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- 172.21.15.40:0/2267969657 shutdown_connections 2024-04-02T16:11:36.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- 172.21.15.40:0/2267969657 wait complete. 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 Processor -- start 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- start start 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 0x7f1ab4142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab414b630 0x7f1ab41e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40b4990 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40b4810 con 0x7f1ab414b630 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1abbbe9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40b4df0 con 0x7f1ab4141c30 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab9985640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab9985640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37050/0 (socket says 172.21.15.40:37050) 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.737+0000 7f1ab9985640 1 -- 172.21.15.40:0/202691218 learned_addr learned my addr 172.21.15.40:0/202691218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aba186640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab414b630 0x7f1ab41e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9184640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 0x7f1ab4142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 -- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 msgr2=0x7f1ab4142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 0x7f1ab4142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 -- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab414b630 msgr2=0x7f1ab41e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab414b630 0x7f1ab41e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 -- 172.21.15.40:0/202691218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab414c590 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9184640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 0x7f1ab4142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9985640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000c370 tx=0x7f1ab000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00052d0 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0005470 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab4144490 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0007260 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab4147db0 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab40685f0 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1ab0007400 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 0x7f1a9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1ab0085d40 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9184640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 0x7f1a9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:36.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.741+0000 7f1ab9184640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 0x7f1a9806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40ad000 tx=0x7f1aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.745+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f1ab0056390 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.949+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1ab4144800 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.953+0000 7f1aaa7fc640 1 -- 172.21.15.40:0/202691218 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+168092 (secure 0 0 0) 0x7f1ab0055c90 con 0x7f1ab40bd9a0 2024-04-02T16:11:36.958 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 msgr2=0x7f1a9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 0x7f1a9806a840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40ad000 tx=0x7f1aa4008040 comp rx=0 tx=0).stop 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 msgr2=0x7f1ab4147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000c370 tx=0x7f1ab000c830 comp rx=0 tx=0).stop 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 shutdown_connections 2024-04-02T16:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4141c30 0x7f1ab4142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a98068380 0x7f1a9806a840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab414b630 0x7f1ab41e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 --2- 172.21.15.40:0/202691218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab40bd9a0 0x7f1ab4147460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 >> 172.21.15.40:0/202691218 conn(0x7f1ab4073310 msgr2=0x7f1ab40b9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 shutdown_connections 2024-04-02T16:11:36.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:36.957+0000 7f1abbbe9640 1 -- 172.21.15.40:0/202691218 wait complete. 2024-04-02T16:11:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1155217760},{"type":"v1","addr":"172.21.15.204:6833","nonce":1155217760}]},"active_addr":"172.21.15.204:6833/1155217760","active_change":"2024-04-02T16:11:11.832309+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2248545051}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":1227860924}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":1403392000}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.204:0","nonce":2907527006}]}]} 2024-04-02T16:11:36.979 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-02T16:11:36.979 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-02T16:11:36.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 Processor -- start 2024-04-02T16:11:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 -- start start 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f05600af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05600ae830 con 0x7f05600b4370 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056006cc80 con 0x7f05600b3800 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f05678cf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f05600b2b50 con 0x7f056006c8a0 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0565e6c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f05600b4370 0x7f056014f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50802/0 (socket says 172.21.15.40:50802) 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0565e6c640 1 -- 172.21.15.40:0/3238050126 learned_addr learned my addr 172.21.15.40:0/3238050126 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f05600af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454744659 0 0) 0x7f05600ae830 con 0x7f05600b4370 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f054c005740 con 0x7f05600b4370 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 105797479 0 0) 0x7f054c005740 con 0x7f05600b4370 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 msgr2=0x7f05600af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f05600af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 >> v1:172.21.15.204:6789/0 conn(0x7f05600b3800 legacy=0x7f05600b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600adfa0 con 0x7f05600b4370 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f0558001a20 con 0x7f05600b4370 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 >> v1:172.21.15.40:6789/0 conn(0x7f05600b4370 legacy=0x7f056014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f054c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c006a20 0x7f054c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f054c0074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05600b2b50 con 0x7f054c005c00 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f054c00dda0 con 0x7f054c006a20 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f054c00dfa0 con 0x7f056006c8a0 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f054c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0565e6c640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f054c0074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c006a20 0x7f054c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 -- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 msgr2=0x7f054c0074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f056006c8a0 0x7f054c0074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 -- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c006a20 msgr2=0x7f054c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c006a20 0x7f054c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f056566b640 1 -- 172.21.15.40:0/3238050126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c005740 con 0x7f054c005c00 2024-04-02T16:11:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.085+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c006a20 0x7f054c006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f054c006020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f055c009ae0 tx=0x7f055c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afce79b97a1df8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0096b0 con 0x7f054c005c00 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055c0a6450 con 0x7f054c005c00 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0557fff640 1 -- 172.21.15.40:0/3238050126 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a6840 con 0x7f054c005c00 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- 172.21.15.40:0/3238050126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 msgr2=0x7f054c006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 --2- 172.21.15.40:0/3238050126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f054c006020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f055c009ae0 tx=0x7f055c01ca70 comp rx=0 tx=0).stop 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0565e6c640 1 -- 172.21.15.40:0/3238050126 reap_dead start 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- 172.21.15.40:0/3238050126 shutdown_connections 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- 172.21.15.40:0/3238050126 >> 172.21.15.40:0/3238050126 conn(0x7f05600b8fd0 msgr2=0x7f05600b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- 172.21.15.40:0/3238050126 shutdown_connections 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- 172.21.15.40:0/3238050126 wait complete. 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 Processor -- start 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- start start 2024-04-02T16:11:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 0x7f05601df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0560070b60 0x7f056009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05600b5650 con 0x7f054c005c00 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05600b54d0 con 0x7f0560070b60 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f05678cf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f056006ced0 con 0x7f0560144f10 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37060/0 (socket says 172.21.15.40:37060) 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 -- 172.21.15.40:0/3688253025 learned_addr learned my addr 172.21.15.40:0/3688253025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 0x7f05601df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 -- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 msgr2=0x7f05601df7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 0x7f05601df7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f056566b640 1 -- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0560070b60 msgr2=0x7f056009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.089+0000 7f0565e6c640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0560070b60 0x7f056009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f056566b640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0560070b60 0x7f056009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f056566b640 1 -- 172.21.15.40:0/3688253025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05601219f0 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 0x7f05601df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f056566b640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c098260 tx=0x7f054c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0a6c00 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055c0a6da0 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05601df290 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c004260 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0560120b10 con 0x7f054c005c00 2024-04-02T16:11:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05601452f0 con 0x7f054c005c00 2024-04-02T16:11:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f055c003620 con 0x7f054c005c00 2024-04-02T16:11:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f055c1209f0 con 0x7f054c005c00 2024-04-02T16:11:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.093+0000 7f0564e6a640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 0x7f054406a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f056009c6a0 tx=0x7f0550009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.097+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f055c0b6070 con 0x7f054c005c00 2024-04-02T16:11:37.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.261+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05600685f0 con 0x7f054c005c00 2024-04-02T16:11:37.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.261+0000 7f0555ffb640 1 -- 172.21.15.40:0/3688253025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f055c0ac020 con 0x7f054c005c00 2024-04-02T16:11:37.267 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:37.268 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 msgr2=0x7f054406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 0x7f054406a840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f056009c6a0 tx=0x7f0550009040 comp rx=0 tx=0).stop 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 msgr2=0x7f05601db710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f055c098260 tx=0x7f054c007a50 comp rx=0 tx=0).stop 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 shutdown_connections 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0560144f10 0x7f05601df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0544068380 0x7f054406a840 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0560070b60 0x7f056009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 --2- 172.21.15.40:0/3688253025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c005c00 0x7f05601db710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 >> 172.21.15.40:0/3688253025 conn(0x7f05600b8fd0 msgr2=0x7f05600672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 shutdown_connections 2024-04-02T16:11:37.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.265+0000 7f05678cf640 1 -- 172.21.15.40:0/3688253025 wait complete. 2024-04-02T16:11:37.287 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-02T16:11:37.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 Processor -- start 2024-04-02T16:11:37.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 -- start start 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7de4073230 0x7f7de40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de40b19e0 con 0x7f7de40b23d0 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7de41497c0 con 0x7f7de4073230 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7deb066640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7de40b05c0 con 0x7f7de4148e50 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7de4073230 0x7f7de40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de8e02640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7de40b23d0 0x7f7de409daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50806/0 (socket says 172.21.15.40:50806) 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de8e02640 1 -- 172.21.15.40:0/355169844 learned_addr learned my addr 172.21.15.40:0/355169844 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de9603640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de3fff640 1 -- 172.21.15.40:0/355169844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 msgr2=0x7f7de4149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de3fff640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de3fff640 1 -- 172.21.15.40:0/355169844 >> v1:172.21.15.40:6789/0 conn(0x7f7de40b23d0 legacy=0x7f7de409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de3fff640 1 -- 172.21.15.40:0/355169844 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de40b08a0 con 0x7f7de4073230 2024-04-02T16:11:37.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.409+0000 7f7de37fe640 1 -- 172.21.15.40:0/355169844 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047252981 0 0) 0x7f7de40b19e0 con 0x7f7de40b23d0 2024-04-02T16:11:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7de3fff640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de40b27b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cc802fd2b6aa6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7de37fe640 1 -- 172.21.15.40:0/355169844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab430 con 0x7f7de4073230 2024-04-02T16:11:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7de37fe640 1 -- 172.21.15.40:0/355169844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd80b0070 con 0x7f7de4073230 2024-04-02T16:11:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7de37fe640 1 -- 172.21.15.40:0/355169844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab7f0 con 0x7f7de4073230 2024-04-02T16:11:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- 172.21.15.40:0/355169844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 msgr2=0x7f7de40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de40b27b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c410 comp rx=0 tx=0).stop 2024-04-02T16:11:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- 172.21.15.40:0/355169844 shutdown_connections 2024-04-02T16:11:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 --2- 172.21.15.40:0/355169844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de40b27b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- 172.21.15.40:0/355169844 >> 172.21.15.40:0/355169844 conn(0x7f7de40b8fd0 msgr2=0x7f7de40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- 172.21.15.40:0/355169844 shutdown_connections 2024-04-02T16:11:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- 172.21.15.40:0/355169844 wait complete. 2024-04-02T16:11:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 Processor -- start 2024-04-02T16:11:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.413+0000 7f7deb066640 1 -- start start 2024-04-02T16:11:37.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7de40b23d0 0x7f7de41de060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4152810 con 0x7f7de4073230 2024-04-02T16:11:37.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7de4152690 con 0x7f7de40b23d0 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7de4152990 con 0x7f7de4148e50 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37066/0 (socket says 172.21.15.40:37066) 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 -- 172.21.15.40:0/1881083488 learned_addr learned my addr 172.21.15.40:0/1881083488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de3fff640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7de40b23d0 0x7f7de41de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de9603640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 -- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 msgr2=0x7f7de4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 -- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7de40b23d0 msgr2=0x7f7de41de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7de40b23d0 0x7f7de41de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 -- 172.21.15.40:0/1881083488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de41e16e0 con 0x7f7de4073230 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de8e02640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc001570 tx=0x7f7ddc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc018070 con 0x7f7de4073230 2024-04-02T16:11:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc00a5e0 con 0x7f7de4073230 2024-04-02T16:11:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de41e19c0 con 0x7f7de4073230 2024-04-02T16:11:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc00a9d0 con 0x7f7de4073230 2024-04-02T16:11:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de41408c0 con 0x7f7de4073230 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de40685f0 con 0x7f7de4073230 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ddc01b070 con 0x7f7de4073230 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 0x7f7dd006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ddc085fc0 con 0x7f7de4073230 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.417+0000 7f7de3fff640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 0x7f7dd006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.421+0000 7f7de3fff640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 0x7f7dd006a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80af000 tx=0x7f7dd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.421+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7ddc056610 con 0x7f7de4073230 2024-04-02T16:11:37.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.581+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7de40b1690 con 0x7f7de4073230 2024-04-02T16:11:37.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7de17fa640 1 -- 172.21.15.40:0/1881083488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f7ddc055f10 con 0x7f7de4073230 2024-04-02T16:11:37.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:37.589 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:37.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 msgr2=0x7f7dd006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 0x7f7dd006a840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80af000 tx=0x7f7dd8008040 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 msgr2=0x7f7de413a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc001570 tx=0x7f7ddc00cd20 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 shutdown_connections 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7de4148e50 0x7f7de4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7dd0068380 0x7f7dd006a840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7de40b23d0 0x7f7de41de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 --2- 172.21.15.40:0/1881083488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de4073230 0x7f7de413a3e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 >> 172.21.15.40:0/1881083488 conn(0x7f7de40b8fd0 msgr2=0x7f7de406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 shutdown_connections 2024-04-02T16:11:37.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.585+0000 7f7deb066640 1 -- 172.21.15.40:0/1881083488 wait complete. 2024-04-02T16:11:37.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-02T16:11:37.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-02T16:11:37.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-02T16:11:37.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-02T16:11:37.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-02T16:11:37.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-02T16:11:37.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-02T16:11:37.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-02T16:11:37.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 Processor -- start 2024-04-02T16:11:37.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 -- start start 2024-04-02T16:11:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f179c0af170 0x7f179c0ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179c0b0210 con 0x7f179c0b50e0 2024-04-02T16:11:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179c0b0760 con 0x7f179c144430 2024-04-02T16:11:37.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a3bff640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f179c0b5890 con 0x7f179c0af170 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a119a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f179c0b50e0 0x7f179c143d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50818/0 (socket says 172.21.15.40:50818) 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a119a640 1 -- 172.21.15.40:0/4226363450 learned_addr learned my addr 172.21.15.40:0/4226363450 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a199b640 1 --2- 172.21.15.40:0/4226363450 >> v2:172.21.15.204:3300/0 conn(0x7f179c0af170 0x7f179c0ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a0999640 1 -- 172.21.15.40:0/4226363450 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3151676738 0 0) 0x7f179c0b0210 con 0x7f179c0b50e0 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a0999640 1 -- 172.21.15.40:0/4226363450 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1788003530 con 0x7f179c0b50e0 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a199b640 1 -- 172.21.15.40:0/4226363450 >> v1:172.21.15.204:6789/0 conn(0x7f179c144430 legacy=0x7f179c14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a199b640 1 -- 172.21.15.40:0/4226363450 >> v1:172.21.15.40:6789/0 conn(0x7f179c0b50e0 legacy=0x7f179c143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.769+0000 7f17a199b640 1 -- 172.21.15.40:0/4226363450 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c0b5ae0 con 0x7f179c0af170 2024-04-02T16:11:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a199b640 1 --2- 172.21.15.40:0/4226363450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c0ae190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f179400d1e0 tx=0x7f179401ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b3065c7603937 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a0999640 1 -- 172.21.15.40:0/4226363450 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940ad070 con 0x7f179c0af170 2024-04-02T16:11:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a0999640 1 -- 172.21.15.40:0/4226363450 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17940a7720 con 0x7f179c0af170 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a0999640 1 -- 172.21.15.40:0/4226363450 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c001680 con 0x7f179c0af170 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 -- 172.21.15.40:0/4226363450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 msgr2=0x7f179c0ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 --2- 172.21.15.40:0/4226363450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c0ae190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f179400d1e0 tx=0x7f179401ea90 comp rx=0 tx=0).stop 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 -- 172.21.15.40:0/4226363450 shutdown_connections 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 --2- 172.21.15.40:0/4226363450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c0ae190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 -- 172.21.15.40:0/4226363450 >> 172.21.15.40:0/4226363450 conn(0x7f179c06c490 msgr2=0x7f179c06c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 -- 172.21.15.40:0/4226363450 shutdown_connections 2024-04-02T16:11:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.773+0000 7f17a3bff640 1 -- 172.21.15.40:0/4226363450 wait complete. 2024-04-02T16:11:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 Processor -- start 2024-04-02T16:11:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 -- start start 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c13b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 0x7f179c141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f179c1416c0 0x7f179c1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f179c152050 con 0x7f179c144430 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f179c151ed0 con 0x7f179c0af170 2024-04-02T16:11:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f179c1521d0 con 0x7f179c1416c0 2024-04-02T16:11:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 0x7f179c141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 0x7f179c141000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37074/0 (socket says 172.21.15.40:37074) 2024-04-02T16:11:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 -- 172.21.15.40:0/2755862203 learned_addr learned my addr 172.21.15.40:0/2755862203 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a219c640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f179c1416c0 0x7f179c1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a199b640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c13b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 -- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f179c1416c0 msgr2=0x7f179c1425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f179c1416c0 0x7f179c1425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 -- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 msgr2=0x7f179c13b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c13b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c141b50 con 0x7f179c144430 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a199b640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f179c0af170 0x7f179c13b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a119a640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 0x7f179c141000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cac0 tx=0x7f178c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c017070 con 0x7f179c144430 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c005cf0 con 0x7f179c144430 2024-04-02T16:11:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c007270 con 0x7f179c144430 2024-04-02T16:11:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.777+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c142ae0 con 0x7f179c144430 2024-04-02T16:11:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c139a30 con 0x7f179c144430 2024-04-02T16:11:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f179c04f870 con 0x7f179c144430 2024-04-02T16:11:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f178c01a070 con 0x7f179c144430 2024-04-02T16:11:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 Processor -- start 2024-04-02T16:11:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1780068380 0x7f178006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f178c08f3a0 con 0x7f179c144430 2024-04-02T16:11:37.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f1780070580 0x7f17800729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1780073050 con 0x7f1780070580 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f178c08b060 con 0x7f179c144430 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17a219c640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f1780070580 0x7f17800729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 -- start start 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 0x7f79300ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7930144430 0x7f7930146c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79300b0210 con 0x7f79300af170 2024-04-02T16:11:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f79300b5890 con 0x7f7930144430 2024-04-02T16:11:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f79378a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79300b0480 con 0x7f79300b50e0 2024-04-02T16:11:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f17a199b640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1780068380 0x7f178006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f793563c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f79300af170 0x7f7930143d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50832/0 (socket says 172.21.15.40:50832) 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f793563c640 1 -- 172.21.15.40:0/1750163012 learned_addr learned my addr 172.21.15.40:0/1750163012 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f7934e3b640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 0x7f79300ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.781+0000 7f7935e3d640 1 --2- 172.21.15.40:0/1750163012 >> v2:172.21.15.204:3300/0 conn(0x7f7930144430 0x7f7930146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4263325146 0 0) 0x7f79300b0210 con 0x7f79300af170 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f791c003530 con 0x7f79300af170 2024-04-02T16:11:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f17a219c640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f1780070580 0x7f17800729a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 275388964 0 0) 0x7f791c003530 con 0x7f79300af170 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7935e3d640 1 -- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 msgr2=0x7f79300ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7935e3d640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 0x7f79300ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7935e3d640 1 -- 172.21.15.40:0/1750163012 >> v1:172.21.15.40:6789/0 conn(0x7f79300af170 legacy=0x7f7930143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7935e3d640 1 -- 172.21.15.40:0/1750163012 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300b0760 con 0x7f7930144430 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7934e3b640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 0x7f79300ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:37.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f17a199b640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1780068380 0x7f178006a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17940b2000 tx=0x7f17940a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 Processor -- start 2024-04-02T16:11:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== osd.2 v2:172.21.15.40:6824/658893798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1780073050 con 0x7f1780070580 2024-04-02T16:11:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 -- start start 2024-04-02T16:11:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 Processor -- start 2024-04-02T16:11:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00aedc0 0x7fb5d01441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 0x7fb5d0144700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5d00af1a0 0x7fb5d0144c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5d00af1a0 0x7fb5d0144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 -- start start 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9cb40caca0 0x7f9cb4038160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cb40cb080 0x7f9cb40368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 0x7f9cb40d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb4039c00 con 0x7f9cb40cb080 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb4037280 con 0x7f9cb40caca0 2024-04-02T16:11:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cc0b85640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb4037560 con 0x7f9cb4036df0 2024-04-02T16:11:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5d00af1a0 0x7fb5d0144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37090/0 (socket says 172.21.15.40:37090) 2024-04-02T16:11:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d5193640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 0x7fb5d0144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d5193640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 0x7fb5d0144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55298/0 (socket says 172.21.15.40:55298) 2024-04-02T16:11:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d5994640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00aedc0 0x7fb5d01441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cb40cb080 0x7f9cb40368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9cb40cb080 0x7f9cb40368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37100/0 (socket says 172.21.15.40:37100) 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/2423241635 learned_addr learned my addr 172.21.15.40:0/2423241635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cbad9d640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 0x7f9cb40d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 -- 172.21.15.40:0/3985750847 learned_addr learned my addr 172.21.15.40:0/3985750847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d0148130 con 0x7fb5d00af1a0 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d00af690 con 0x7fb5d00b0c80 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d0147ae0 con 0x7fb5d00aedc0 2024-04-02T16:11:37.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f7935e3d640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7930144430 0x7f7930146c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f792001c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b8c8d5ee325359 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f64a2fc2640 1 Processor -- start 2024-04-02T16:11:37.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/2423241635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 msgr2=0x7f9cb40d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 -- 172.21.15.40:0/3985750847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00aedc0 msgr2=0x7fb5d01441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3985750847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00aedc0 0x7fb5d01441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 -- 172.21.15.40:0/3985750847 >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 msgr2=0x7fb5d0144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3985750847 >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 0x7fb5d0144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7fb5d6195640 1 -- 172.21.15.40:0/3985750847 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d0147dc0 con 0x7fb5d00af1a0 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f64a2fc2640 1 -- start start 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 0x7f9cb40d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/2423241635 >> v2:172.21.15.204:3300/0 conn(0x7f9cb40caca0 msgr2=0x7f9cb4038160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/2423241635 >> v2:172.21.15.204:3300/0 conn(0x7f9cb40caca0 0x7f9cb4038160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/2423241635 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb40386a0 con 0x7f9cb40cb080 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.785+0000 7f9cbad9d640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 0x7f9cb40d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200b1070 con 0x7f7930144430 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79200ac440 con 0x7f7930144430 2024-04-02T16:11:37.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f7927fff640 1 -- 172.21.15.40:0/1750163012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c004e70 con 0x7f7930144430 2024-04-02T16:11:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 -- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7930144430 msgr2=0x7f7930146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7930144430 0x7f7930146c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f792001c3b0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a2fc2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649c0b2030 0x7f649c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a2fc2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c06cfe0 con 0x7f649c146ba0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a2fc2640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f649c073160 con 0x7f649c06c3e0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a2fc2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f649c06ca20 con 0x7f649c0b2030 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f649c06c3e0 0x7f649c0b1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:41430/0 (socket says 172.21.15.40:41430) 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 -- 172.21.15.40:0/288428200 learned_addr learned my addr 172.21.15.40:0/288428200 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339929652 0 0) 0x7f649c073160 con 0x7f649c06c3e0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f648800bd40 con 0x7f649c06c3e0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2547218659 0 0) 0x7f648800bd40 con 0x7f649c06c3e0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649c0b2030 msgr2=0x7f649c0af7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649c0b2030 0x7f649c0af7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 >> v1:172.21.15.40:6789/0 conn(0x7f649c146ba0 legacy=0x7f649c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c06cd00 con 0x7f649c06c3e0 2024-04-02T16:11:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f6494001a00 con 0x7f649c06c3e0 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 >> v1:172.21.15.204:6789/0 conn(0x7f649c06c3e0 legacy=0x7f649c0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f648800c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648800d020 0x7f648800d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f648800dc30 0x7f64880094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f649c06ca20 con 0x7f648800dc30 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6488009dc0 con 0x7f648800d020 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6488009fc0 con 0x7f648800c240 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f648800c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 -- 172.21.15.40:0/1750163012 shutdown_connections 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7930144430 0x7f7930146c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 --2- 172.21.15.40:0/1750163012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79300b50e0 0x7f79300ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 -- 172.21.15.40:0/1750163012 >> 172.21.15.40:0/1750163012 conn(0x7f793006c490 msgr2=0x7f793006c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a17bf640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648800d020 0x7f648800d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a27c1640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f648800dc30 0x7f64880094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb40368b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007540 tx=0x7f9cac01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59a45e5706072e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 -- 172.21.15.40:0/1750163012 shutdown_connections 2024-04-02T16:11:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f79378a0640 1 -- 172.21.15.40:0/1750163012 wait complete. 2024-04-02T16:11:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 -- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648800d020 msgr2=0x7f648800d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f648800d020 0x7f648800d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 -- 172.21.15.40:0/288428200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f648800dc30 msgr2=0x7f64880094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f9cb959a640 1 -- 172.21.15.40:0/2423241635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0b0070 con 0x7f9cb40cb080 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f648800dc30 0x7f64880094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a1fc0640 1 -- 172.21.15.40:0/288428200 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648800bd40 con 0x7f648800c240 2024-04-02T16:11:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f64a27c1640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f648800dc30 0x7f64880094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f9cb959a640 1 -- 172.21.15.40:0/2423241635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0ab600 con 0x7f9cb40cb080 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.789+0000 7f9cb959a640 1 -- 172.21.15.40:0/2423241635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0ab9d0 con 0x7f9cb40cb080 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- 172.21.15.40:0/2423241635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 msgr2=0x7f9cb40368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb40368b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007540 tx=0x7f9cac01c540 comp rx=0 tx=0).stop 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- 172.21.15.40:0/2423241635 shutdown_connections 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb4036df0 0x7f9cb40d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- 172.21.15.40:0/2423241635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb40368b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- 172.21.15.40:0/2423241635 >> v2:172.21.15.204:3300/0 conn(0x7f9cb40caca0 0x7f9cb4038160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- 172.21.15.40:0/2423241635 >> 172.21.15.40:0/2423241635 conn(0x7f9cb403eaa0 msgr2=0x7f9cb403eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- 172.21.15.40:0/2423241635 shutdown_connections 2024-04-02T16:11:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 Processor -- start 2024-04-02T16:11:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3985750847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00af1a0 0x7fb5d0144c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b7870 tx=0x7fb5c8020d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bb8b6c0c90e3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 Processor -- start 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d4992640 1 -- 172.21.15.40:0/3985750847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80a9930 con 0x7fb5d00af1a0 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d4992640 1 -- 172.21.15.40:0/3985750847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c80a9ad0 con 0x7fb5d00af1a0 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d4992640 1 -- 172.21.15.40:0/3985750847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80b13f0 con 0x7fb5d00af1a0 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- 172.21.15.40:0/2423241635 wait complete. 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 Processor -- start 2024-04-02T16:11:37.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 -- start start 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79300af170 0x7f793013e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930144430 0x7f793006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 0x7f7930138be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7930150930 con 0x7f7930144430 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79301507b0 con 0x7f79300af170 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f79378a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7930150ab0 con 0x7f7930145790 2024-04-02T16:11:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 -- start start 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f648800c640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f649400a7a0 tx=0x7f649401d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14957952704aa01d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 0x7f7930138be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7934e3b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930144430 0x7f793006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 0x7f7930138be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41620/0 (socket says 172.21.15.40:41620) 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 -- 172.21.15.40:0/2190511895 learned_addr learned my addr 172.21.15.40:0/2190511895 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7934e3b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930144430 0x7f793006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37124/0 (socket says 172.21.15.40:37124) 2024-04-02T16:11:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f793563c640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79300af170 0x7f793013e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb3406c610 0x7ffb340ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb340aef50 0x7ffb3406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00af1a0 msgr2=0x7fb5d0144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 --2- 172.21.15.40:0/3985750847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00af1a0 0x7fb5d0144c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b7870 tx=0x7fb5c8020d80 comp rx=0 tx=0).stop 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 shutdown_connections 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 --2- 172.21.15.40:0/3985750847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00af1a0 0x7fb5d0144c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 --2- 172.21.15.40:0/3985750847 >> v2:172.21.15.204:3300/0 conn(0x7fb5d00b0c80 0x7fb5d0144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 --2- 172.21.15.40:0/3985750847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00aedc0 0x7fb5d01441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 >> 172.21.15.40:0/3985750847 conn(0x7fb5d00ba110 msgr2=0x7fb5d00ba520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 shutdown_connections 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- 172.21.15.40:0/3985750847 wait complete. 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3406c9f0 con 0x7ffb340b3580 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3406d8a0 con 0x7ffb340aef50 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3b538640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3406daf0 con 0x7ffb3406c610 2024-04-02T16:11:37.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3ad37640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ffb340b3580 0x7ffb340b39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:41436/0 (socket says 172.21.15.40:41436) 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3ad37640 1 -- 172.21.15.40:0/372066334 learned_addr learned my addr 172.21.15.40:0/372066334 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400a3f0 con 0x7f648800c240 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 -- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79300af170 msgr2=0x7f793013e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79300af170 0x7f793013e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 -- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930144430 msgr2=0x7f793006f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7930144430 0x7f793006f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 -- 172.21.15.40:0/2190511895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79300726f0 con 0x7f7930145790 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb39d35640 1 --2- 172.21.15.40:0/372066334 >> v2:172.21.15.40:3300/0 conn(0x7ffb340aef50 0x7ffb3406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- start start 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cb40caca0 0x7f9cb400d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb4175ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb400c830 0x7f9cb40bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40d4bf0 con 0x7f9cb40cb080 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb40d4a70 con 0x7f9cb40caca0 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cc0b85640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb40d4d70 con 0x7f9cb400c830 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb3a536640 1 --2- 172.21.15.40:0/372066334 >> v2:172.21.15.204:3300/0 conn(0x7ffb3406c610 0x7ffb340ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940a9070 con 0x7f648800c240 2024-04-02T16:11:37.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f64a0fbe640 1 -- 172.21.15.40:0/288428200 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940ae6b0 con 0x7f648800c240 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cbad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb400c830 0x7f9cb40bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cbad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb400c830 0x7f9cb40bff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41630/0 (socket says 172.21.15.40:41630) 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cbad9d640 1 -- 172.21.15.40:0/1797362590 learned_addr learned my addr 172.21.15.40:0/1797362590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb4175ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7f7935e3d640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 0x7f7930138be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f7920019150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb39d35640 1 -- 172.21.15.40:0/372066334 >> v2:172.21.15.204:3300/0 conn(0x7ffb3406c610 msgr2=0x7ffb340ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb39d35640 1 --2- 172.21.15.40:0/372066334 >> v2:172.21.15.204:3300/0 conn(0x7ffb3406c610 0x7ffb340ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb39d35640 1 -- 172.21.15.40:0/372066334 >> v1:172.21.15.204:6789/0 conn(0x7ffb340b3580 legacy=0x7ffb340b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7ffb39d35640 1 -- 172.21.15.40:0/372066334 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb340af460 con 0x7ffb340aef50 2024-04-02T16:11:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 Processor -- start 2024-04-02T16:11:37.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.793+0000 7fb5d6996640 1 -- start start 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb400c830 msgr2=0x7f9cb40bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9cb400c830 0x7f9cb40bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cb40caca0 msgr2=0x7f9cb400d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cb40caca0 0x7f9cb400d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb400cd10 con 0x7f9cb40cb080 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 msgr2=0x7f648800c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 --2- 172.21.15.40:0/288428200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f648800c640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f649400a7a0 tx=0x7f649401d2e0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a27c1640 1 -- 172.21.15.40:0/288428200 reap_dead start 2024-04-02T16:11:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cb9d9b640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb4175ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0ab000 tx=0x7f9cac01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- 172.21.15.40:0/288428200 shutdown_connections 2024-04-02T16:11:37.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- 172.21.15.40:0/288428200 >> 172.21.15.40:0/288428200 conn(0x7f649c0b8d50 msgr2=0x7f649c0b9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- 172.21.15.40:0/288428200 shutdown_connections 2024-04-02T16:11:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0ba020 con 0x7f9cb40cb080 2024-04-02T16:11:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0aa7d0 con 0x7f9cb40cb080 2024-04-02T16:11:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0b0040 con 0x7f9cb40cb080 2024-04-02T16:11:37.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- 172.21.15.40:0/288428200 wait complete. 2024-04-02T16:11:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cc0b85640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb41756e0 con 0x7f9cb40cb080 2024-04-02T16:11:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cc0b85640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb40cde50 con 0x7f9cb40cb080 2024-04-02T16:11:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f9cc0b85640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9cb4005820 con 0x7f9cb40cb080 2024-04-02T16:11:37.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200ba020 con 0x7f7930145790 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79200b6480 con 0x7f7930145790 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7930071c10 con 0x7f7930145790 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200b1040 con 0x7f7930145790 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7930145d00 con 0x7f7930145790 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d00aedc0 0x7fb5d00b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00af1a0 0x7fb5d00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 0x7fb5d00b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d006c120 con 0x7fb5d00b0c80 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d006bfa0 con 0x7fb5d00aedc0 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6996640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d006c2a0 con 0x7fb5d00af1a0 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 0x7fb5d00b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 0x7fb5d00b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37158/0 (socket says 172.21.15.40:37158) 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 -- 172.21.15.40:0/3012768498 learned_addr learned my addr 172.21.15.40:0/3012768498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d5193640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00af1a0 0x7fb5d00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d5994640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d00aedc0 0x7fb5d00b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 -- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00af1a0 msgr2=0x7fb5d00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d00af1a0 0x7fb5d00b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 -- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d00aedc0 msgr2=0x7fb5d00b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d00aedc0 0x7fb5d00b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d00b3870 con 0x7fb5d00b0c80 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5d6195640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 0x7fb5d00b2d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5c801f430 tx=0x7fb5c801db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80a9930 con 0x7fb5d00b0c80 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c8007bb0 con 0x7fb5d00b0c80 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 Processor -- start 2024-04-02T16:11:37.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- start start 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7ffb39d35640 1 --2- 172.21.15.40:0/372066334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3406f5f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb3406de30 tx=0x7ffb3001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5508e008213c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f649c1e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649c0723a0 0x7f649c139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649c145ec0 0x7f649c13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f649c0b5340 con 0x7f649c0723a0 2024-04-02T16:11:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f649c0b51c0 con 0x7f649c145ec0 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f64a2fc2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f649c1486b0 con 0x7f648800c240 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a17bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649c0723a0 0x7f649c139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a17bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649c0723a0 0x7f649c139980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37160/0 (socket says 172.21.15.40:37160) 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f649c1e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a17bf640 1 -- 172.21.15.40:0/3966452350 learned_addr learned my addr 172.21.15.40:0/3966452350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a27c1640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649c145ec0 0x7f649c13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9cac0bd020 con 0x7f9cb40cb080 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c9c068450 0x7f9c9c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9cac1213a0 con 0x7f9cb40cb080 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7f9c9c0706d0 0x7f9c9c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9cac00af30 con 0x7f9c9c0706d0 2024-04-02T16:11:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9cac121790 con 0x7f9cb40cb080 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79200bd020 con 0x7f7930145790 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7914068400 0x7f791406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79201200e0 con 0x7f7930145790 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f793563c640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7914068400 0x7f791406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7ffb39534640 1 -- 172.21.15.40:0/372066334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300ab680 con 0x7ffb340aef50 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7ffb39534640 1 -- 172.21.15.40:0/372066334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb300b0070 con 0x7ffb340aef50 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7ffb39534640 1 -- 172.21.15.40:0/372066334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300aba40 con 0x7ffb340aef50 2024-04-02T16:11:37.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- 172.21.15.40:0/372066334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 msgr2=0x7ffb3406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- 172.21.15.40:0/372066334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3406f5f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb3406de30 tx=0x7ffb3001c480 comp rx=0 tx=0).stop 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- 172.21.15.40:0/372066334 shutdown_connections 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- 172.21.15.40:0/372066334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3406f5f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- 172.21.15.40:0/372066334 >> v2:172.21.15.204:3300/0 conn(0x7ffb3406c610 0x7ffb340ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- 172.21.15.40:0/372066334 >> 172.21.15.40:0/372066334 conn(0x7ffb340b8d50 msgr2=0x7ffb340b9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- 172.21.15.40:0/372066334 shutdown_connections 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.797+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8009510 con 0x7fb5d00b0c80 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5d6996640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d00b2a60 con 0x7fb5d00b0c80 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5d6996640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d00b32a0 con 0x7fb5d00b0c80 2024-04-02T16:11:37.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 -- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649c145ec0 msgr2=0x7f649c13a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649c145ec0 0x7f649c13a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 -- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649c0723a0 msgr2=0x7f649c139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649c0723a0 0x7f649c139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c146840 con 0x7f648800c240 2024-04-02T16:11:37.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9cbad9d640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7f9c9c0706d0 0x7f9c9c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9cba59c640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c9c068450 0x7f9c9c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9cbad9d640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7f9c9c0706d0 0x7f9c9c072af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f793563c640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7914068400 0x7f791406a8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f792c000f70 tx=0x7f792c004f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a1fc0640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f649c1e03e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f648800a190 tx=0x7f648800e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- 172.21.15.40:0/372066334 wait complete. 2024-04-02T16:11:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b2030 con 0x7f648800c240 2024-04-02T16:11:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c070060 con 0x7f648800c240 2024-04-02T16:11:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c146560 con 0x7f648800c240 2024-04-02T16:11:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940ae8a0 con 0x7f648800c240 2024-04-02T16:11:37.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 Processor -- start 2024-04-02T16:11:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940aec50 con 0x7f648800c240 2024-04-02T16:11:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9cba59c640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c9c068450 0x7f9c9c06a910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00096f0 tx=0x7f9cb0006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6484001040 con 0x7f648800c240 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== osd.1 v2:172.21.15.40:6816/422638552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9cac00af30 con 0x7f9c9c0706d0 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f79378a0640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f79300b50e0 0x7f7930139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 --> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79300b87c0 con 0x7f79300b50e0 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f7934e3b640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f79300b50e0 0x7f7930139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7f7934e3b640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f79300b50e0 0x7f7930139120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- start start 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 0x7ffb34150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3409ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb340b3580 0x7ffb340b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34159190 con 0x7ffb340aef50 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34159010 con 0x7ffb3406c610 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7ffb3b538640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb34159310 con 0x7ffb340b3580 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5d6996640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5d00b2890 con 0x7fb5d00b0c80 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb5c8012030 con 0x7fb5d00b0c80 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5b40683d0 0x7fb5b406a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb5c81218a0 con 0x7fb5d00b0c80 2024-04-02T16:11:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7fb5b4070650 0x7fb5b4072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5c80a9ad0 con 0x7fb5b4070650 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.801+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb5c81262a0 con 0x7fb5d00b0c80 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7fb5d5994640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5b40683d0 0x7fb5b406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7fb5d5193640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7fb5b4070650 0x7fb5b4072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 0x7ffb34150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 0x7ffb34150d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55348/0 (socket says 172.21.15.40:55348) 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 -- 172.21.15.40:0/3439238734 learned_addr learned my addr 172.21.15.40:0/3439238734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb39d35640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3409ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3ad37640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb340b3580 0x7ffb340b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 -- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb340b3580 msgr2=0x7ffb340b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb340b3580 0x7ffb340b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 -- 172.21.15.40:0/3439238734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 msgr2=0x7ffb3409ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb340aef50 0x7ffb3409ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3409b1d0 con 0x7ffb3406c610 2024-04-02T16:11:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7fb5d5994640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5b40683d0 0x7fb5b406a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000a8d0 tx=0x7fb5c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7fb5d5193640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7fb5b4070650 0x7fb5b4072a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7ffb3a536640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 0x7ffb34150d30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400d800 tx=0x7ffb2400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== osd.3 v2:172.21.15.40:6800/1014552006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79300b87c0 con 0x7f79300b50e0 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6494007070 con 0x7f648800c240 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64800683d0 0x7f648006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6494121270 con 0x7f648800c240 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f6480070650 0x7f6480072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6494006e50 con 0x7f6480070650 2024-04-02T16:11:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6494121750 con 0x7f648800c240 2024-04-02T16:11:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64a17bf640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64800683d0 0x7f648006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64a27c1640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f6480070650 0x7f6480072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64a27c1640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f6480070650 0x7f6480072a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== osd.6 v2:172.21.15.204:6803/1944425714 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5c80a9ad0 con 0x7fb5b4070650 2024-04-02T16:11:37.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.805+0000 7f64a17bf640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64800683d0 0x7f648006a890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f649c13dda0 tx=0x7f6498009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400b6c0 con 0x7ffb3406c610 2024-04-02T16:11:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb340b1c50 con 0x7ffb3406c610 2024-04-02T16:11:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb340b2210 con 0x7ffb3406c610 2024-04-02T16:11:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb24019070 con 0x7ffb3406c610 2024-04-02T16:11:37.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 Processor -- start 2024-04-02T16:11:37.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400ba60 con 0x7ffb3406c610 2024-04-02T16:11:37.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb1c001040 con 0x7ffb3406c610 2024-04-02T16:11:37.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 -- start start 2024-04-02T16:11:37.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 0x7f7d9c14f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d9c14f570 0x7f7d9c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9c14c9f0 con 0x7f7d9c0b3550 2024-04-02T16:11:37.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d9c0b1800 con 0x7f7d9c14f570 2024-04-02T16:11:37.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7da0a4c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9c14bf10 con 0x7f7d9c072770 2024-04-02T16:11:37.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== osd.0 v2:172.21.15.40:6801/519999405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6494006e50 con 0x7f6480070650 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7d93fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d9c14f570 0x7f7d9c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7d9ad9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f7d9c0b3550 0x7f7d9c070e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:41452/0 (socket says 172.21.15.40:41452) 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7d9ad9d640 1 -- 172.21.15.40:0/2318393576 learned_addr learned my addr 172.21.15.40:0/2318393576 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.809+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2318393576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 0x7f7d9c14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d99d9b640 1 -- 172.21.15.40:0/2318393576 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964070403 0 0) 0x7f7d9c14c9f0 con 0x7f7d9c0b3550 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d99d9b640 1 -- 172.21.15.40:0/2318393576 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d88003530 con 0x7f7d9c0b3550 2024-04-02T16:11:37.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffb2400bc00 con 0x7ffb3406c610 2024-04-02T16:11:37.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d9a59c640 1 -- 172.21.15.40:0/2318393576 >> v2:172.21.15.204:3300/0 conn(0x7f7d9c14f570 msgr2=0x7f7d9c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2318393576 >> v2:172.21.15.204:3300/0 conn(0x7f7d9c14f570 0x7f7d9c0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d9a59c640 1 -- 172.21.15.40:0/2318393576 >> v1:172.21.15.204:6789/0 conn(0x7f7d9c0b3550 legacy=0x7f7d9c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f7d9a59c640 1 -- 172.21.15.40:0/2318393576 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c14c1f0 con 0x7f7d9c072770 2024-04-02T16:11:37.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffb180683d0 0x7ffb1806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffb24086db0 con 0x7ffb3406c610 2024-04-02T16:11:37.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7ffb180705f0 0x7ffb18072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb24001d40 con 0x7ffb180705f0 2024-04-02T16:11:37.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffb24087290 con 0x7ffb3406c610 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb39d35640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffb180683d0 0x7ffb1806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb3ad37640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7ffb180705f0 0x7ffb18072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb3ad37640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7ffb180705f0 0x7ffb18072a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7ffb39d35640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffb180683d0 0x7ffb1806a890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb300af000 tx=0x7ffb300a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 --> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f179c13b2d0 con 0x7f1780070580 2024-04-02T16:11:37.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.813+0000 7f17927fc640 1 -- 172.21.15.40:0/2755862203 <== osd.2 v2:172.21.15.40:6824/658893798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f179c13b2d0 con 0x7f1780070580 2024-04-02T16:11:37.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== osd.4 v2:172.21.15.204:6800/3004039088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb24001d40 con 0x7ffb180705f0 2024-04-02T16:11:37.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2318393576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 0x7f7d9c14f030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d8401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6400f231646a4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f1780070580 msgr2=0x7f17800729a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f1780070580 0x7f17800729a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7d99d9b640 1 -- 172.21.15.40:0/2318393576 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840b0070 con 0x7f7d9c072770 2024-04-02T16:11:37.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7d99d9b640 1 -- 172.21.15.40:0/2318393576 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840ab600 con 0x7f7d9c072770 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1780068380 msgr2=0x7f178006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1780068380 0x7f178006a840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17940b2000 tx=0x7f17940a9070 comp rx=0 tx=0).stop 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 msgr2=0x7f179c141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 --2- 172.21.15.40:0/2755862203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f179c144430 0x7f179c141000 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f178c00cac0 tx=0x7f178c00cf80 comp rx=0 tx=0).stop 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a219c640 1 -- 172.21.15.40:0/2755862203 reap_dead start 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 shutdown_connections 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 >> 172.21.15.40:0/2755862203 conn(0x7f179c06c490 msgr2=0x7f179c0ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 shutdown_connections 2024-04-02T16:11:37.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f17a3bff640 1 -- 172.21.15.40:0/2755862203 wait complete. 2024-04-02T16:11:37.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7d99d9b640 1 -- 172.21.15.40:0/2318393576 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d840ab9d0 con 0x7f7d9c072770 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2318393576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 msgr2=0x7f7d9c14f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2318393576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 0x7f7d9c14f030 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097c0 tx=0x7f7d8401c350 comp rx=0 tx=0).stop 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2318393576 shutdown_connections 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2318393576 >> v2:172.21.15.204:3300/0 conn(0x7f7d9c14f570 0x7f7d9c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2318393576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c072770 0x7f7d9c14f030 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2318393576 >> 172.21.15.40:0/2318393576 conn(0x7f7d9c0b7f50 msgr2=0x7f7d9c0b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2318393576 shutdown_connections 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2318393576 wait complete. 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 Processor -- start 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.817+0000 7f7da0a4c640 1 -- start start 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c072770 0x7f7d9c09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9c0b3550 0x7f7d9c06cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c14f570 0x7f7d9c09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c14bc40 con 0x7f7d9c072770 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c14bac0 con 0x7f7d9c0b3550 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7da0a4c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9c072b50 con 0x7f7d9c14f570 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d9ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c14f570 0x7f7d9c09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d9ad9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c14f570 0x7f7d9c09ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41680/0 (socket says 172.21.15.40:41680) 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d9ad9d640 1 -- 172.21.15.40:0/2031420684 learned_addr learned my addr 172.21.15.40:0/2031420684 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9c0b3550 0x7f7d9c06cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9cc0b85640 1 -- 172.21.15.40:0/1797362590 --> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9cb40db1d0 con 0x7f9c9c0706d0 2024-04-02T16:11:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c072770 0x7f7d9c09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9caaffd640 1 -- 172.21.15.40:0/1797362590 <== osd.1 v2:172.21.15.40:6816/422638552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9cb40db1d0 con 0x7f9c9c0706d0 2024-04-02T16:11:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7f9c9c0706d0 msgr2=0x7f9c9c072af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7f9c9c0706d0 0x7f9c9c072af0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c9c068450 msgr2=0x7f9c9c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c9c068450 0x7f9c9c06a910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00096f0 tx=0x7f9cb0006c50 comp rx=0 tx=0).stop 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 msgr2=0x7f9cb4175ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 --2- 172.21.15.40:0/1797362590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cb40cb080 0x7f9cb4175ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0ab000 tx=0x7f9cac01ad80 comp rx=0 tx=0).stop 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9cbad9d640 1 -- 172.21.15.40:0/1797362590 reap_dead start 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 shutdown_connections 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 >> 172.21.15.40:0/1797362590 conn(0x7f9cb403eaa0 msgr2=0x7f9cb40d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 shutdown_connections 2024-04-02T16:11:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f9ca8ff9640 1 -- 172.21.15.40:0/1797362590 wait complete. 2024-04-02T16:11:37.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 -- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c14f570 msgr2=0x7f7d9c09ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d9c14f570 0x7f7d9c09ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 -- 172.21.15.40:0/2031420684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c072770 msgr2=0x7f7d9c09b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c072770 0x7f7d9c09b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c09af50 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.821+0000 7f7d93fff640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9c0b3550 0x7f7d9c06cb70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e870 tx=0x7f7d8c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.825+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c017070 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.825+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c09d230 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.825+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c09b1d0 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.825+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8c00ee60 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c00cb60 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d80001040 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7d8c01a070 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d7c068450 0x7f7d7c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d8c087890 con 0x7f7d9c0b3550 2024-04-02T16:11:37.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7f7d7c0706d0 0x7f7d7c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d8c006880 con 0x7f7d7c0706d0 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7d8c087c80 con 0x7f7d9c0b3550 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d7c068450 0x7f7d7c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7f7d9ad9d640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7f7d7c0706d0 0x7f7d7c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7f7d9a59c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d7c068450 0x7f7d7c06a910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d840ad000 tx=0x7f7d840a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7f7d9ad9d640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7f7d7c0706d0 0x7f7d7c072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7faf1229f640 1 Processor -- start 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.829+0000 7faf1229f640 1 -- start start 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1229f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faf04037800 0x7faf04035d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1229f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 0x7faf040e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1229f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf040ea450 con 0x7faf040eade0 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1229f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7faf04039c90 con 0x7faf040ea970 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1229f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7faf040ea120 con 0x7faf04037800 2024-04-02T16:11:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1129d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faf04037800 0x7faf04035d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf11a9e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faf040eade0 0x7faf040e42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50848/0 (socket says 172.21.15.40:50848) 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf11a9e640 1 -- 172.21.15.40:0/1966702273 learned_addr learned my addr 172.21.15.40:0/1966702273 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf10a9c640 1 --2- 172.21.15.40:0/1966702273 >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 0x7faf040e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1129d640 1 -- 172.21.15.40:0/1966702273 >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 msgr2=0x7faf040e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faefffff640 1 -- 172.21.15.40:0/1966702273 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4271815800 0 0) 0x7faf040ea450 con 0x7faf040eade0 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1129d640 1 --2- 172.21.15.40:0/1966702273 >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 0x7faf040e3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1129d640 1 -- 172.21.15.40:0/1966702273 >> v1:172.21.15.40:6789/0 conn(0x7faf040eade0 legacy=0x7faf040e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf1129d640 1 -- 172.21.15.40:0/1966702273 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf04038390 con 0x7faf04037800 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7faf10a9c640 1 --2- 172.21.15.40:0/1966702273 >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 0x7faf040e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.833+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== osd.5 v2:172.21.15.204:6801/246318220 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d8c006880 con 0x7f7d7c0706d0 2024-04-02T16:11:37.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 --> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79300aecf0 con 0x7f79300b50e0 2024-04-02T16:11:37.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1129d640 1 --2- 172.21.15.40:0/1966702273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf04035d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf04039f00 tx=0x7faf0001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db123f7d29c703fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f7925ffb640 1 -- 172.21.15.40:0/2190511895 <== osd.3 v2:172.21.15.40:6800/1014552006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79300aecf0 con 0x7f79300b50e0 2024-04-02T16:11:37.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7fb5d6996640 1 -- 172.21.15.40:0/3012768498 --> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5d00b15f0 con 0x7fb5b4070650 2024-04-02T16:11:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f79300b50e0 msgr2=0x7f7930139120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f79300b50e0 0x7f7930139120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7fb5c67fc640 1 -- 172.21.15.40:0/3012768498 <== osd.6 v2:172.21.15.204:6803/1944425714 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb5d00b15f0 con 0x7fb5b4070650 2024-04-02T16:11:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7914068400 msgr2=0x7f791406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7914068400 0x7f791406a8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f792c000f70 tx=0x7f792c004f30 comp rx=0 tx=0).stop 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 msgr2=0x7f7930138be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 --2- 172.21.15.40:0/2190511895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7930145790 0x7f7930138be0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79200097c0 tx=0x7f7920019150 comp rx=0 tx=0).stop 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f7935e3d640 1 -- 172.21.15.40:0/2190511895 reap_dead start 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 shutdown_connections 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 >> 172.21.15.40:0/2190511895 conn(0x7f793006c490 msgr2=0x7f793013f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 shutdown_connections 2024-04-02T16:11:37.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7f79378a0640 1 -- 172.21.15.40:0/2190511895 wait complete. 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faefffff640 1 -- 172.21.15.40:0/1966702273 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000b0070 con 0x7faf04037800 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faefffff640 1 -- 172.21.15.40:0/1966702273 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf000ab600 con 0x7faf04037800 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faefffff640 1 -- 172.21.15.40:0/1966702273 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000ab9d0 con 0x7faf04037800 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 -- 172.21.15.40:0/1966702273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 msgr2=0x7faf04035d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 --2- 172.21.15.40:0/1966702273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf04035d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faf04039f00 tx=0x7faf0001c410 comp rx=0 tx=0).stop 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 -- 172.21.15.40:0/1966702273 shutdown_connections 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 --2- 172.21.15.40:0/1966702273 >> v2:172.21.15.40:3300/0 conn(0x7faf040ea970 0x7faf040e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 --2- 172.21.15.40:0/1966702273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf04035d60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 -- 172.21.15.40:0/1966702273 >> 172.21.15.40:0/1966702273 conn(0x7faf0403a860 msgr2=0x7faf0403ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 -- 172.21.15.40:0/1966702273 shutdown_connections 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.837+0000 7faf1229f640 1 -- 172.21.15.40:0/1966702273 wait complete. 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 Processor -- start 2024-04-02T16:11:37.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 -- start start 2024-04-02T16:11:37.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7fb5b4070650 msgr2=0x7fb5b4072a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7fb5b4070650 0x7fb5b4072a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5b40683d0 msgr2=0x7fb5b406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5b40683d0 0x7fb5b406a890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000a8d0 tx=0x7fb5c0008040 comp rx=0 tx=0).stop 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 msgr2=0x7fb5d00b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 --2- 172.21.15.40:0/3012768498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d00b0c80 0x7fb5d00b2d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb5c801f430 tx=0x7fb5c801db20 comp rx=0 tx=0).stop 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5d6195640 1 -- 172.21.15.40:0/3012768498 reap_dead start 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 shutdown_connections 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 >> 172.21.15.40:0/3012768498 conn(0x7fb5d00ba110 msgr2=0x7fb5d00b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 shutdown_connections 2024-04-02T16:11:37.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7fb5b3fff640 1 -- 172.21.15.40:0/3012768498 wait complete. 2024-04-02T16:11:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2024-04-02T16:11:37.873 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-02T16:11:37.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 --> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6484002d60 con 0x7f6480070650 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf040ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf040ea970 0x7faf040ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf040eade0 0x7faf04193d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7faf040ef400 con 0x7faf040ea970 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faf040ef280 con 0x7faf04037800 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1229f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7faf040ef580 con 0x7faf040eade0 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf10a9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf040ea970 0x7faf040ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf10a9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf040ea970 0x7faf040ed8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37192/0 (socket says 172.21.15.40:37192) 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf10a9c640 1 -- 172.21.15.40:0/2401396252 learned_addr learned my addr 172.21.15.40:0/2401396252 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf1129d640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf040ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf040eade0 0x7faf04193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 -- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 msgr2=0x7faf040ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf04037800 0x7faf040ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 -- 172.21.15.40:0/2401396252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf040ea970 msgr2=0x7faf040ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf040ea970 0x7faf040ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0402a550 con 0x7faf040eade0 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.841+0000 7faf11a9e640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf040eade0 0x7faf04193d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faf0c050e60 tx=0x7faf0c02d5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c04fde0 con 0x7faf040eade0 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7f64927fc640 1 -- 172.21.15.40:0/3966452350 <== osd.0 v2:172.21.15.40:6801/519999405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6484002d60 con 0x7f6480070650 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c06cae0 con 0x7faf040eade0 2024-04-02T16:11:37.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c083420 con 0x7faf040eade0 2024-04-02T16:11:37.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0402a830 con 0x7faf040eade0 2024-04-02T16:11:37.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf040ec6b0 con 0x7faf040eade0 2024-04-02T16:11:37.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.845+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf040053a0 con 0x7faf040eade0 2024-04-02T16:11:37.876 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:37.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 --> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb1c002de0 con 0x7ffb180705f0 2024-04-02T16:11:37.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-02T16:11:37.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7faf0c0835c0 con 0x7faf040eade0 2024-04-02T16:11:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faef0068380 0x7faef006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7ffb2affd640 1 -- 172.21.15.40:0/3439238734 <== osd.4 v2:172.21.15.204:6800/3004039088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffb1c002de0 con 0x7ffb180705f0 2024-04-02T16:11:37.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf0c0f03a0 con 0x7faf040eade0 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7faef0070520 0x7faef0072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faef0072ff0 con 0x7faef0070520 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faf0c0f0830 con 0x7faf040eade0 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f6480070650 msgr2=0x7f6480072a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f6480070650 0x7f6480072a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64800683d0 msgr2=0x7f648006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64800683d0 0x7f648006a890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f649c13dda0 tx=0x7f6498009040 comp rx=0 tx=0).stop 2024-04-02T16:11:37.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 msgr2=0x7f649c1e03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 --2- 172.21.15.40:0/3966452350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f648800c240 0x7f649c1e03e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f648800a190 tx=0x7f648800e050 comp rx=0 tx=0).stop 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a27c1640 1 -- 172.21.15.40:0/3966452350 reap_dead start 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 shutdown_connections 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 >> 172.21.15.40:0/3966452350 conn(0x7f649c0b8d50 msgr2=0x7f649c0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faf10a9c640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7faef0070520 0x7faef0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faf1129d640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faef0068380 0x7faef006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:37.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 shutdown_connections 2024-04-02T16:11:37.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7f64a2fc2640 1 -- 172.21.15.40:0/3966452350 wait complete. 2024-04-02T16:11:37.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faf10a9c640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7faef0070520 0x7faef0072940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faf1129d640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faef0068380 0x7faef006a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faf000ad000 tx=0x7faf000a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:37.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.849+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== osd.7 v2:172.21.15.204:6802/176733231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faef0072ff0 con 0x7faef0070520 2024-04-02T16:11:37.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7ffb180705f0 msgr2=0x7ffb18072a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7ffb180705f0 0x7ffb18072a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffb180683d0 msgr2=0x7ffb1806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffb180683d0 0x7ffb1806a890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb300af000 tx=0x7ffb300a7000 comp rx=0 tx=0).stop 2024-04-02T16:11:37.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 msgr2=0x7ffb34150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 --2- 172.21.15.40:0/3439238734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb3406c610 0x7ffb34150d30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400d800 tx=0x7ffb2400dcc0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3ad37640 1 -- 172.21.15.40:0/3439238734 reap_dead start 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 shutdown_connections 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 >> 172.21.15.40:0/3439238734 conn(0x7ffb340b8d50 msgr2=0x7ffb340b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 shutdown_connections 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.857+0000 7ffb3b538640 1 -- 172.21.15.40:0/3439238734 wait complete. 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 --> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d80002d60 con 0x7f7d7c0706d0 2024-04-02T16:11:37.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7d92ffd640 1 -- 172.21.15.40:0/2031420684 <== osd.5 v2:172.21.15.204:6801/246318220 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d80002d60 con 0x7f7d7c0706d0 2024-04-02T16:11:37.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7f7d7c0706d0 msgr2=0x7f7d7c072af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7f7d7c0706d0 0x7f7d7c072af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d7c068450 msgr2=0x7f7d7c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d7c068450 0x7f7d7c06a910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7d840ad000 tx=0x7f7d840a6000 comp rx=0 tx=0).stop 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9c0b3550 msgr2=0x7f7d9c06cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 --2- 172.21.15.40:0/2031420684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d9c0b3550 0x7f7d9c06cb70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e870 tx=0x7f7d8c00ed30 comp rx=0 tx=0).stop 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7d9ad9d640 1 -- 172.21.15.40:0/2031420684 reap_dead start 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 shutdown_connections 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 >> 172.21.15.40:0/2031420684 conn(0x7f7d9c0b7f50 msgr2=0x7f7d9c14aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 shutdown_connections 2024-04-02T16:11:37.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.861+0000 7f7da0a4c640 1 -- 172.21.15.40:0/2031420684 wait complete. 2024-04-02T16:11:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-04-02T16:11:37.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-02T16:11:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:37.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-02T16:11:37.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 --> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf040ec880 con 0x7faef0070520 2024-04-02T16:11:37.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faefdffb640 1 -- 172.21.15.40:0/2401396252 <== osd.7 v2:172.21.15.204:6802/176733231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf040ec880 con 0x7faef0070520 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7faef0070520 msgr2=0x7faef0072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7faef0070520 0x7faef0072940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faef0068380 msgr2=0x7faef006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7faef0068380 0x7faef006a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faf000ad000 tx=0x7faf000a6000 comp rx=0 tx=0).stop 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf040eade0 msgr2=0x7faf04193d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 --2- 172.21.15.40:0/2401396252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf040eade0 0x7faf04193d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faf0c050e60 tx=0x7faf0c02d5e0 comp rx=0 tx=0).stop 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf11a9e640 1 -- 172.21.15.40:0/2401396252 reap_dead start 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 shutdown_connections 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 >> 172.21.15.40:0/2401396252 conn(0x7faf0403a860 msgr2=0x7faf040e8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:37.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 shutdown_connections 2024-04-02T16:11:37.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:37.873+0000 7faf1229f640 1 -- 172.21.15.40:0/2401396252 wait complete. 2024-04-02T16:11:37.890 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:37.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T16:11:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:37.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-02T16:11:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:37.893 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-02T16:11:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-04-02T16:11:37.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-02T16:11:38.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 Processor -- start 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 -- start start 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff7f00ae6f0 0x7ff7f0146f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 0x7ff7f0143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f01478e0 con 0x7ff7f0144220 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f006cc20 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f744d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f00af910 con 0x7ff7f0147450 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f5c4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 0x7ff7f0143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f6c4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f6c4c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37202/0 (socket says 172.21.15.40:37202) 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f5c4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 0x7ff7f0143ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41696/0 (socket says 172.21.15.40:41696) 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.052+0000 7ff7f5c4a640 1 -- 172.21.15.40:0/3226054886 learned_addr learned my addr 172.21.15.40:0/3226054886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 --2- 172.21.15.40:0/3226054886 >> v2:172.21.15.204:3300/0 conn(0x7ff7f00ae6f0 0x7ff7f0146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 -- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 msgr2=0x7ff7f0143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 --2- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 0x7ff7f0143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 -- 172.21.15.40:0/3226054886 >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 msgr2=0x7ff7f00b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 --2- 172.21.15.40:0/3226054886 >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 -- 172.21.15.40:0/3226054886 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f00afbf0 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f6c4c640 1 --2- 172.21.15.40:0/3226054886 >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:38.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.056+0000 7ff7f644b640 1 --2- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f0146f10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7f002d1b0 tx=0x7ff7ec01d360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4726fab383bd05c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f5449640 1 -- 172.21.15.40:0/3226054886 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0b1070 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f5449640 1 -- 172.21.15.40:0/3226054886 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec008be0 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f5449640 1 -- 172.21.15.40:0/3226054886 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00a0d0 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f00ae6f0 msgr2=0x7ff7f0146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f0146f10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7f002d1b0 tx=0x7ff7ec01d360 comp rx=0 tx=0).stop 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- 172.21.15.40:0/3226054886 shutdown_connections 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- 172.21.15.40:0/3226054886 >> v2:172.21.15.40:3300/0 conn(0x7ff7f0144220 0x7ff7f00b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0147450 0x7ff7f0143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- 172.21.15.40:0/3226054886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f0146f10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- 172.21.15.40:0/3226054886 >> 172.21.15.40:0/3226054886 conn(0x7ff7f00b8b80 msgr2=0x7ff7f00b8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- 172.21.15.40:0/3226054886 shutdown_connections 2024-04-02T16:11:38.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- 172.21.15.40:0/3226054886 wait complete. 2024-04-02T16:11:38.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 Processor -- start 2024-04-02T16:11:38.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- start start 2024-04-02T16:11:38.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 Processor -- start 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 -- start start 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9d48037110 0x7f9d480dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4dd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9d480d15b0 0x7f9d48036a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50850/0 (socket says 172.21.15.40:50850) 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4dd9b640 1 -- 172.21.15.40:0/194608935 learned_addr learned my addr 172.21.15.40:0/194608935 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4ed9d640 1 --2- 172.21.15.40:0/194608935 >> v2:172.21.15.204:3300/0 conn(0x7f9d48037110 0x7f9d480dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480d09e0 con 0x7f9d480d15b0 2024-04-02T16:11:38.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d48038440 con 0x7f9d4803b7f0 2024-04-02T16:11:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 Processor -- start 2024-04-02T16:11:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f01e3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 0x7ff7f01dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 0x7ff7f01e0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01446b0 con 0x7ff7f00ae6f0 2024-04-02T16:11:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00af590 con 0x7ff7f0147450 2024-04-02T16:11:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7ff7f744d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f00730c0 con 0x7ff7f0144220 2024-04-02T16:11:38.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.060+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9d48038720 con 0x7f9d48037110 2024-04-02T16:11:38.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 Processor -- start 2024-04-02T16:11:38.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 -- start start 2024-04-02T16:11:38.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f5c4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 0x7ff7f01dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 0x7ff7f01e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f5c4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 0x7ff7f01dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41712/0 (socket says 172.21.15.40:41712) 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f5c4a640 1 -- 172.21.15.40:0/570487484 learned_addr learned my addr 172.21.15.40:0/570487484 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f644b640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f01e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110217580 0 0) 0x7f9d44001120 con 0x7f9d480d15b0 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d38003530 con 0x7f9d480d15b0 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248584284 0 0) 0x7f9d48038440 con 0x7f9d4803b7f0 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d44001120 con 0x7f9d4803b7f0 2024-04-02T16:11:38.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4ed9d640 1 -- 172.21.15.40:0/194608935 >> v1:172.21.15.204:6789/0 conn(0x7f9d4803b7f0 legacy=0x7f9d480d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4ed9d640 1 -- 172.21.15.40:0/194608935 >> v1:172.21.15.40:6789/0 conn(0x7f9d480d15b0 legacy=0x7f9d48036a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4ed9d640 1 -- 172.21.15.40:0/194608935 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4803c190 con 0x7f9d48037110 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 -- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 msgr2=0x7ff7f01dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 0x7ff7f01dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 -- 172.21.15.40:0/570487484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 msgr2=0x7ff7f01e3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f01e3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 -- 172.21.15.40:0/570487484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f0071ea0 con 0x7ff7f0147450 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f644b640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f01e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f6c4c640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 0x7ff7f01e0270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000e930 tx=0x7ff7e000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d7800c360 0x7f7d7801dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d780cd420 0x7f7d780c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7801da80 con 0x7f7d7803c230 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7801e3e0 con 0x7f7d780cd420 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d86cf6640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d780cc850 con 0x7f7d7800c360 2024-04-02T16:11:38.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d780cd420 0x7f7d780c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7d780cd420 0x7f7d780c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37224/0 (socket says 172.21.15.40:37224) 2024-04-02T16:11:38.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 -- 172.21.15.40:0/262843802 learned_addr learned my addr 172.21.15.40:0/262843802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d84a92640 1 --2- 172.21.15.40:0/262843802 >> v2:172.21.15.204:3300/0 conn(0x7f7d7800c360 0x7f7d7801dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 -- start start 2024-04-02T16:11:38.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 -- 172.21.15.40:0/262843802 >> v2:172.21.15.204:3300/0 conn(0x7f7d7800c360 msgr2=0x7f7d7801dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 --2- 172.21.15.40:0/262843802 >> v2:172.21.15.204:3300/0 conn(0x7f7d7800c360 0x7f7d7801dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 -- 172.21.15.40:0/262843802 >> v1:172.21.15.204:6789/0 conn(0x7f7d7803c230 legacy=0x7f7d780ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f7d85293640 1 -- 172.21.15.40:0/262843802 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7800c740 con 0x7f7d780cd420 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0017070 con 0x7ff7f0147450 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f744d640 1 -- 172.21.15.40:0/570487484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f00b0e70 con 0x7ff7f0147450 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f744d640 1 -- 172.21.15.40:0/570487484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f0072140 con 0x7ff7f0147450 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e0004020 con 0x7ff7f0147450 2024-04-02T16:11:38.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0004410 con 0x7ff7f0147450 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc428036c60 0x7fc4280370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc42802aca0 con 0x7fc428037c20 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4280367d0 con 0x7fc428036c60 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc436780640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc428038000 con 0x7fc4280c9700 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc43577e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc428037c20 0x7fc4280c2e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50866/0 (socket says 172.21.15.40:50866) 2024-04-02T16:11:38.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc43577e640 1 -- 172.21.15.40:0/3474466771 learned_addr learned my addr 172.21.15.40:0/3474466771 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7ff7f744d640 1 -- 172.21.15.40:0/570487484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d80052a0 con 0x7ff7f0147450 2024-04-02T16:11:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3474466771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc435f7f640 1 --2- 172.21.15.40:0/3474466771 >> v2:172.21.15.40:3300/0 conn(0x7fc428036c60 0x7fc4280370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f2454dbe640 1 Processor -- start 2024-04-02T16:11:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f2454dbe640 1 -- start start 2024-04-02T16:11:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fdef201c640 1 Processor -- start 2024-04-02T16:11:38.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 -- start start 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc434f7d640 1 -- 172.21.15.40:0/3474466771 >> v2:172.21.15.40:3300/0 conn(0x7fc428036c60 msgr2=0x7fc4280370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7fc427fff640 1 -- 172.21.15.40:0/3474466771 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2944952115 0 0) 0x7fc42802aca0 con 0x7fc428037c20 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3474466771 >> v2:172.21.15.40:3300/0 conn(0x7fc428036c60 0x7fc4280370f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc434f7d640 1 -- 172.21.15.40:0/3474466771 >> v1:172.21.15.40:6789/0 conn(0x7fc428037c20 legacy=0x7fc4280c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc434f7d640 1 -- 172.21.15.40:0/3474466771 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc428037630 con 0x7fc4280c9700 2024-04-02T16:11:38.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.064+0000 7f9d4ed9d640 1 --2- 172.21.15.40:0/194608935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d48037110 0x7f9d480dc2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000b70 tx=0x7f9d50097a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2543e594844e88c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f2454dbe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24500b6080 0x7f24500b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f2454dbe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24500b44f0 0x7f245014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d85293640 1 --2- 172.21.15.40:0/262843802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c9400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7d80051210 tx=0x7f7d800a0190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f7db320ed34e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff7e000c940 con 0x7ff7f0147450 2024-04-02T16:11:38.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f2454dbe640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2450072d50 con 0x7f2450072970 2024-04-02T16:11:38.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7ff7e6ffd640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24500b44f0 0x7f245014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24500b44f0 0x7f245014eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37234/0 (socket says 172.21.15.40:37234) 2024-04-02T16:11:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 -- 172.21.15.40:0/1896524099 learned_addr learned my addr 172.21.15.40:0/1896524099 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2450144c30 con 0x7f24500b44f0 2024-04-02T16:11:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f24501445e0 con 0x7f24500b6080 2024-04-02T16:11:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff7e0086e30 con 0x7ff7f0147450 2024-04-02T16:11:38.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d7f7fe640 1 -- 172.21.15.40:0/262843802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80134070 con 0x7f7d780cd420 2024-04-02T16:11:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d7f7fe640 1 -- 172.21.15.40:0/262843802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d8012f640 con 0x7f7d780cd420 2024-04-02T16:11:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d7f7fe640 1 -- 172.21.15.40:0/262843802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8012fab0 con 0x7f7d780cd420 2024-04-02T16:11:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdee4035940 0x7fdee4036540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 -- 172.21.15.40:0/1896524099 >> v2:172.21.15.204:3300/0 conn(0x7f24500b6080 msgr2=0x7f24500b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7ff7f644b640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244dd9b640 1 --2- 172.21.15.40:0/1896524099 >> v2:172.21.15.204:3300/0 conn(0x7f24500b6080 0x7f24500b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee402a020 con 0x7fdee4035560 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdee4037430 con 0x7fdee4035940 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef201c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdee40362f0 con 0x7fdee40c8600 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef181b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef181b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55430/0 (socket says 172.21.15.40:55430) 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef181b640 1 -- 172.21.15.40:0/2882239913 learned_addr learned my addr 172.21.15.40:0/2882239913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 -- 172.21.15.40:0/262843802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 msgr2=0x7f7d780c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 --2- 172.21.15.40:0/262843802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c9400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7d80051210 tx=0x7f7d800a0190 comp rx=0 tx=0).stop 2024-04-02T16:11:38.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef0819640 1 --2- 172.21.15.40:0/2882239913 >> v2:172.21.15.40:3300/0 conn(0x7fdee4035940 0x7fdee4036540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 -- 172.21.15.40:0/262843802 shutdown_connections 2024-04-02T16:11:38.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 --2- 172.21.15.40:0/262843802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c9400 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 --2- 172.21.15.40:0/262843802 >> v2:172.21.15.204:3300/0 conn(0x7f7d7800c360 0x7f7d7801dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 -- 172.21.15.40:0/262843802 >> 172.21.15.40:0/262843802 conn(0x7f7d7800b5e0 msgr2=0x7f7d7800b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7ff7f644b640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 0x7ff7d406a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ae000 tx=0x7ff7ec0a8000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 -- 172.21.15.40:0/262843802 shutdown_connections 2024-04-02T16:11:38.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f7d86cf6640 1 -- 172.21.15.40:0/262843802 wait complete. 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef0819640 1 -- 172.21.15.40:0/2882239913 >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 msgr2=0x7fdee40d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef0819640 1 --2- 172.21.15.40:0/2882239913 >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef0819640 1 -- 172.21.15.40:0/2882239913 >> v1:172.21.15.204:6789/0 conn(0x7fdee4035560 legacy=0x7fdee4033800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef0819640 1 -- 172.21.15.40:0/2882239913 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee4033f10 con 0x7fdee4035940 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fdef181b640 1 --2- 172.21.15.40:0/2882239913 >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:38.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3474466771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c9b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097c0 tx=0x7fc42001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6678cb22eedf5279 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 --2- 172.21.15.40:0/1896524099 >> v2:172.21.15.204:3300/0 conn(0x7f24500b6080 0x7f24500b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 -- 172.21.15.40:0/1896524099 >> v1:172.21.15.204:6789/0 conn(0x7f2450072970 legacy=0x7f24500b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f244ed9d640 1 -- 172.21.15.40:0/1896524099 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24501448c0 con 0x7f24500b44f0 2024-04-02T16:11:38.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f5692200640 1 Processor -- start 2024-04-02T16:11:38.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7f5692200640 1 -- start start 2024-04-02T16:11:38.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc427fff640 1 -- 172.21.15.40:0/3474466771 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200b0070 con 0x7fc4280c9700 2024-04-02T16:11:38.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc427fff640 1 -- 172.21.15.40:0/3474466771 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4200ab600 con 0x7fc4280c9700 2024-04-02T16:11:38.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.068+0000 7fc427fff640 1 -- 172.21.15.40:0/3474466771 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4200ab9d0 con 0x7fc4280c9700 2024-04-02T16:11:38.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 Processor -- start 2024-04-02T16:11:38.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- start start 2024-04-02T16:11:38.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f5692200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 0x7f568c070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f5692200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f568c141390 0x7f568c1417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f5692200640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568c141d20 con 0x7f568c073070 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 0x7f568c070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f5692200640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f568c072680 con 0x7f568c0b6e90 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 0x7f568c070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37248/0 (socket says 172.21.15.40:37248) 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568b7fe640 1 -- 172.21.15.40:0/4142051376 learned_addr learned my addr 172.21.15.40:0/4142051376 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f568c0728b0 con 0x7f568c141390 2024-04-02T16:11:38.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f244ed9d640 1 --2- 172.21.15.40:0/1896524099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24500b44f0 0x7f245014eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24500b7710 tx=0x7f244801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a7d221650d5b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f244d59a640 1 -- 172.21.15.40:0/1896524099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480a5590 con 0x7f24500b44f0 2024-04-02T16:11:38.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568bfff640 1 --2- 172.21.15.40:0/4142051376 >> v2:172.21.15.204:3300/0 conn(0x7f568c141390 0x7f568c1417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- 172.21.15.40:0/3474466771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 msgr2=0x7fc4280c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- 172.21.15.40:0/3474466771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c9b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4200097c0 tx=0x7fc42001c380 comp rx=0 tx=0).stop 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- 172.21.15.40:0/3474466771 shutdown_connections 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- 172.21.15.40:0/3474466771 >> v2:172.21.15.40:3300/0 conn(0x7fc428036c60 0x7fc4280370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- 172.21.15.40:0/3474466771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c9b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- 172.21.15.40:0/3474466771 >> 172.21.15.40:0/3474466771 conn(0x7fc4280086c0 msgr2=0x7fc42803eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d7800c360 0x7f7d781548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d7803c230 0x7f7d780377c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78028b80 con 0x7f7d780cd420 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d78028a00 con 0x7f7d7800c360 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d78028d00 con 0x7f7d7803c230 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- 172.21.15.40:0/3474466771 shutdown_connections 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37252/0 (socket says 172.21.15.40:37252) 2024-04-02T16:11:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 -- 172.21.15.40:0/4154905898 learned_addr learned my addr 172.21.15.40:0/4154905898 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f244d59a640 1 -- 172.21.15.40:0/1896524099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24480a5730 con 0x7f24500b44f0 2024-04-02T16:11:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f244d59a640 1 -- 172.21.15.40:0/1896524099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480a5b00 con 0x7f24500b44f0 2024-04-02T16:11:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568a7fc640 1 -- 172.21.15.40:0/4142051376 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608429981 0 0) 0x7f568c141d20 con 0x7f568c073070 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568a7fc640 1 -- 172.21.15.40:0/4142051376 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5674003530 con 0x7f568c073070 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568bfff640 1 -- 172.21.15.40:0/4142051376 >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 msgr2=0x7f568c070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568bfff640 1 --2- 172.21.15.40:0/4142051376 >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 0x7f568c070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568bfff640 1 -- 172.21.15.40:0/4142051376 >> v1:172.21.15.40:6789/0 conn(0x7f568c073070 legacy=0x7f568c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f568bfff640 1 -- 172.21.15.40:0/4142051376 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0b7270 con 0x7f568c141390 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 -- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d7803c230 msgr2=0x7f7d780377c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d7803c230 0x7f7d780377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 -- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d7800c360 msgr2=0x7f7d781548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d7800c360 0x7f7d781548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 -- 172.21.15.40:0/4154905898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d78020800 con 0x7f7d780cd420 2024-04-02T16:11:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef0819640 1 --2- 172.21.15.40:0/2882239913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035940 0x7fdee4036540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdee402a440 tx=0x7fdee801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de693e80256b4746 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d85293640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d8006add0 tx=0x7f7d80069370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdee3fff640 1 -- 172.21.15.40:0/2882239913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80b0070 con 0x7fdee4035940 2024-04-02T16:11:38.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24500b44f0 msgr2=0x7f245014eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdee3fff640 1 -- 172.21.15.40:0/2882239913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee80ab600 con 0x7fdee4035940 2024-04-02T16:11:38.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdee3fff640 1 -- 172.21.15.40:0/2882239913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80ab9d0 con 0x7fdee4035940 2024-04-02T16:11:38.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1896524099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24500b44f0 0x7f245014eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24500b7710 tx=0x7f244801c8f0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- 172.21.15.40:0/3474466771 wait complete. 2024-04-02T16:11:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 Processor -- start 2024-04-02T16:11:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 -- 172.21.15.40:0/2882239913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035940 msgr2=0x7fdee4036540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 --2- 172.21.15.40:0/2882239913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035940 0x7fdee4036540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdee402a440 tx=0x7fdee801c3f0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8013d070 con 0x7f7d780cd420 2024-04-02T16:11:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d80139680 con 0x7f7d780cd420 2024-04-02T16:11:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80134040 con 0x7f7d780cd420 2024-04-02T16:11:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- 172.21.15.40:0/4154905898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d78036fc0 con 0x7f7d780cd420 2024-04-02T16:11:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7f7d86cf6640 1 -- 172.21.15.40:0/4154905898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d781556a0 con 0x7f7d780cd420 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- start start 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 0x7fc4280b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc428007660 con 0x7fc428036c60 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc4280074e0 con 0x7fc428037c20 2024-04-02T16:11:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fc436780640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc4280077e0 con 0x7fc4280c9700 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 -- 172.21.15.40:0/2882239913 shutdown_connections 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 --2- 172.21.15.40:0/2882239913 >> v2:172.21.15.204:3300/0 conn(0x7fdee40c8600 0x7fdee40d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 --2- 172.21.15.40:0/2882239913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035940 0x7fdee4036540 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.072+0000 7fdef201c640 1 -- 172.21.15.40:0/2882239913 >> 172.21.15.40:0/2882239913 conn(0x7fdee403abd0 msgr2=0x7fdee4005ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc434f7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc434f7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55440/0 (socket says 172.21.15.40:55440) 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc434f7d640 1 -- 172.21.15.40:0/3534185109 learned_addr learned my addr 172.21.15.40:0/3534185109 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 0x7fc4280b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 shutdown_connections 2024-04-02T16:11:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1896524099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24500b44f0 0x7f245014eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc435f7f640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1896524099 >> v2:172.21.15.204:3300/0 conn(0x7f24500b6080 0x7f24500b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 >> 172.21.15.40:0/1896524099 conn(0x7f245006e4f0 msgr2=0x7f245006e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f568bfff640 1 --2- 172.21.15.40:0/4142051376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1417e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f568c147600 tx=0x7f568401c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8987b954353b232 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- 172.21.15.40:0/2882239913 shutdown_connections 2024-04-02T16:11:38.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f7d86cf6640 1 -- 172.21.15.40:0/4154905898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d780057f0 con 0x7f7d780cd420 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 -- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 msgr2=0x7fc4280c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 -- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 msgr2=0x7fc4280c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 -- 172.21.15.40:0/3534185109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4280c0110 con 0x7fc428036c60 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:38.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 shutdown_connections 2024-04-02T16:11:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- 172.21.15.40:0/1896524099 wait complete. 2024-04-02T16:11:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fc43577e640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 0x7fc4280b0310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc42c000c10 tx=0x7fc42c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 Processor -- start 2024-04-02T16:11:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 -- start start 2024-04-02T16:11:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- 172.21.15.40:0/2882239913 wait complete. 2024-04-02T16:11:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7d80140070 con 0x7f7d780cd420 2024-04-02T16:11:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdc940b7250 0x7fdc94145500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f7d7d7fa640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 0x7f7d6406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 0x7fdc9402cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc940b5a50 con 0x7fdc940b4a50 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940b31f0 con 0x7fdc940b7250 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc9b083640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdc940b7630 con 0x7fdc9402c6f0 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc99620640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc940b4a50 0x7fdc940b4ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50892/0 (socket says 172.21.15.40:50892) 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc99620640 1 -- 172.21.15.40:0/2014137571 learned_addr learned my addr 172.21.15.40:0/2014137571 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/2014137571 >> v2:172.21.15.40:3300/0 conn(0x7fdc940b7250 0x7fdc94145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d801a4bc0 con 0x7f7d780cd420 2024-04-02T16:11:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc93fff640 1 --2- 172.21.15.40:0/2014137571 >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 0x7fdc9402cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 Processor -- start 2024-04-02T16:11:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- start start 2024-04-02T16:11:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 Processor -- start 2024-04-02T16:11:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f7d84a92640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 0x7f7d6406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24500b44f0 0x7f2450150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 0x7f245013a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f245014f880 con 0x7f2450072970 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24500b4b90 con 0x7f24500b6080 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f2454dbe640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f245014fa00 con 0x7f24500b44f0 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f244e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f244ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 0x7f245013a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f244e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37284/0 (socket says 172.21.15.40:37284) 2024-04-02T16:11:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f244e59c640 1 -- 172.21.15.40:0/1162777069 learned_addr learned my addr 172.21.15.40:0/1162777069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7f244dd9b640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24500b44f0 0x7f2450150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- start start 2024-04-02T16:11:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee4035940 0x7fdee416a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee40c8600 0x7fdee416cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4035db0 con 0x7fdee4035560 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee40c8e20 con 0x7fdee40c8600 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef201c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdee40d5c00 con 0x7fdee4035940 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef101a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef101a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37298/0 (socket says 172.21.15.40:37298) 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdef101a640 1 -- 172.21.15.40:0/1335392428 learned_addr learned my addr 172.21.15.40:0/1335392428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc98e1f640 1 -- 172.21.15.40:0/2014137571 >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 msgr2=0x7fdc9402cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/2014137571 >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 0x7fdc9402cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc98e1f640 1 -- 172.21.15.40:0/2014137571 >> v1:172.21.15.40:6789/0 conn(0x7fdc940b4a50 legacy=0x7fdc940b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc98e1f640 1 -- 172.21.15.40:0/2014137571 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc940b5cc0 con 0x7fdc940b7250 2024-04-02T16:11:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.076+0000 7fdc93fff640 1 --2- 172.21.15.40:0/2014137571 >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 0x7fdc9402cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef181b640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee40c8600 0x7fdee416cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 -- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24500b44f0 msgr2=0x7f2450150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24500b44f0 0x7f2450150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 -- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 msgr2=0x7f245013a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 0x7f245013a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 -- 172.21.15.40:0/1162777069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2450148050 con 0x7f2450072970 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244ed9d640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 0x7f245013a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f7d84a92640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 0x7f7d6406a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d740098f0 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 -- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee4035940 msgr2=0x7fdee416a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee4035940 0x7fdee416a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 -- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee40c8600 msgr2=0x7fdee416cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee40c8600 0x7fdee416cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 -- 172.21.15.40:0/1335392428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee40cac40 con 0x7fdee4035560 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f244e59c640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f244000c530 tx=0x7f244000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef101a640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c500 tx=0x7fdedc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/2014137571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc94145500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc940b2560 tx=0x7fdc8801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4725ae42d93b830 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0074e0 con 0x7fc428036c60 2024-04-02T16:11:38.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc937fe640 1 -- 172.21.15.40:0/2014137571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc880b0070 con 0x7fdc940b7250 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4280c64d0 con 0x7fc428036c60 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc016020 con 0x7fdee4035560 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee40c2f70 con 0x7fdee4035560 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4280c0390 con 0x7fc428036c60 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c007680 con 0x7fc428036c60 2024-04-02T16:11:38.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee40c9c40 con 0x7fdee4035560 2024-04-02T16:11:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0049f0 con 0x7fc428036c60 2024-04-02T16:11:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7d8009a0f0 con 0x7f7d780cd420 2024-04-02T16:11:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc937fe640 1 -- 172.21.15.40:0/2014137571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc880ab600 con 0x7fdc940b7250 2024-04-02T16:11:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 -- 172.21.15.40:0/2014137571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 msgr2=0x7fdc94145500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 --2- 172.21.15.40:0/2014137571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc94145500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdc940b2560 tx=0x7fdc8801c3c0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440016020 con 0x7f2450072970 2024-04-02T16:11:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee40053a0 con 0x7fdee4035560 2024-04-02T16:11:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568a7fc640 1 -- 172.21.15.40:0/4142051376 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840b2070 con 0x7f568c141390 2024-04-02T16:11:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50123700 con 0x7f9d48037110 2024-04-02T16:11:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50123900 con 0x7f9d48037110 2024-04-02T16:11:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f9d4d59a640 1 -- 172.21.15.40:0/194608935 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50123cf0 con 0x7f9d48037110 2024-04-02T16:11:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568a7fc640 1 -- 172.21.15.40:0/4142051376 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56840a58e0 con 0x7f568c141390 2024-04-02T16:11:38.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568a7fc640 1 -- 172.21.15.40:0/4142051376 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840a5c90 con 0x7f568c141390 2024-04-02T16:11:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2450147d70 con 0x7f2450072970 2024-04-02T16:11:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2450150200 con 0x7f2450072970 2024-04-02T16:11:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245004f8e0 con 0x7f2450072970 2024-04-02T16:11:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4100052a0 con 0x7fc428036c60 2024-04-02T16:11:38.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff7e0056e10 con 0x7ff7f0147450 2024-04-02T16:11:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 msgr2=0x7f568c1417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- 172.21.15.40:0/4142051376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1417e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f568c147600 tx=0x7f568401c930 comp rx=0 tx=0).stop 2024-04-02T16:11:38.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 shutdown_connections 2024-04-02T16:11:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- 172.21.15.40:0/4142051376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1417e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- 172.21.15.40:0/4142051376 >> v2:172.21.15.40:3300/0 conn(0x7f568c0b6e90 0x7f568c070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 >> 172.21.15.40:0/4142051376 conn(0x7f568c06ca90 msgr2=0x7f568c06cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 shutdown_connections 2024-04-02T16:11:38.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- 172.21.15.40:0/4142051376 wait complete. 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 -- 172.21.15.40:0/2014137571 shutdown_connections 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 --2- 172.21.15.40:0/2014137571 >> v2:172.21.15.204:3300/0 conn(0x7fdc9402c6f0 0x7fdc9402cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 --2- 172.21.15.40:0/2014137571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc94145500 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 -- 172.21.15.40:0/2014137571 >> 172.21.15.40:0/2014137571 conn(0x7fdc940bb3e0 msgr2=0x7fdc940bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 -- 172.21.15.40:0/2014137571 shutdown_connections 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.080+0000 7fdc9b083640 1 -- 172.21.15.40:0/2014137571 wait complete. 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 Processor -- start 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- start start 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940b4a50 0x7fdc94140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc9413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94152620 con 0x7fdc940b7250 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc94145c50 con 0x7fdc9402c6f0 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdc941527a0 con 0x7fdc940b4a50 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55476/0 (socket says 172.21.15.40:55476) 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 -- 172.21.15.40:0/4228219366 learned_addr learned my addr 172.21.15.40:0/4228219366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 Processor -- start 2024-04-02T16:11:38.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- start start 2024-04-02T16:11:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 -- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940b4a50 msgr2=0x7fdc94140a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T16:11:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940b4a50 0x7fdc94140a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 -- 172.21.15.40:0/4228219366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 msgr2=0x7fdc9413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc9413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 -- 172.21.15.40:0/4228219366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc94141460 con 0x7fdc9402c6f0 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c073070 0x7f568c1e3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 0x7f568c0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0b6550 con 0x7f568c073070 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0b63d0 con 0x7f568c141390 2024-04-02T16:11:38.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f5692200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f568c1547a0 con 0x7f568c0b6e90 2024-04-02T16:11:38.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc98e1f640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc880ae040 tx=0x7fdc88019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc880b9020 con 0x7fdc9402c6f0 2024-04-02T16:11:38.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568affd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 0x7f568c0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55478/0 (socket says 172.21.15.40:55478) 2024-04-02T16:11:38.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc880ab9d0 con 0x7fdc9402c6f0 2024-04-02T16:11:38.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 -- 172.21.15.40:0/4294215843 learned_addr learned my addr 172.21.15.40:0/4294215843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568affd640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 0x7f568c0adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41792/0 (socket says 172.21.15.40:41792) 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc880b0040 con 0x7fdc9402c6f0 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc94141740 con 0x7fdc9402c6f0 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 -- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 msgr2=0x7f568c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 0x7f568c0adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 -- 172.21.15.40:0/4294215843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c073070 msgr2=0x7f568c1e3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c073070 0x7f568c1e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7f568bfff640 1 -- 172.21.15.40:0/4294215843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0afd70 con 0x7f568c141390 2024-04-02T16:11:38.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.084+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9413e310 con 0x7fdc9402c6f0 2024-04-02T16:11:38.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f568bfff640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56840b0040 tx=0x7f56840196d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d48037110 msgr2=0x7f9d480dc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/194608935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d48037110 0x7f9d480dc2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000b70 tx=0x7f9d50097a90 comp rx=0 tx=0).stop 2024-04-02T16:11:38.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94068d00 con 0x7fdc9402c6f0 2024-04-02T16:11:38.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 shutdown_connections 2024-04-02T16:11:38.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/194608935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d48037110 0x7f9d480dc2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 >> 172.21.15.40:0/194608935 conn(0x7f9d4803eca0 msgr2=0x7f9d4803f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 shutdown_connections 2024-04-02T16:11:38.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840b2050 con 0x7f568c141390 2024-04-02T16:11:38.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56840be070 con 0x7f568c141390 2024-04-02T16:11:38.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684009450 con 0x7f568c141390 2024-04-02T16:11:38.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- 172.21.15.40:0/194608935 wait complete. 2024-04-02T16:11:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c1e3fc0 con 0x7f568c141390 2024-04-02T16:11:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c0affe0 con 0x7f568c141390 2024-04-02T16:11:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdc880bc020 con 0x7fdc9402c6f0 2024-04-02T16:11:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c0b30a0 con 0x7f568c141390 2024-04-02T16:11:38.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7fdc917fa640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 0x7fdc7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdc88120cc0 con 0x7fdc9402c6f0 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 Processor -- start 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- start start 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d48037110 0x7f9d4816cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480df670 con 0x7f9d48037110 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4803bde0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.088+0000 7f9d4f59e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480df7f0 con 0x7f9d480d15b0 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4ed9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2440005220 con 0x7f2450072970 2024-04-02T16:11:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440007030 con 0x7f2450072970 2024-04-02T16:11:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55488/0 (socket says 172.21.15.40:55488) 2024-04-02T16:11:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 -- 172.21.15.40:0/498773228 learned_addr learned my addr 172.21.15.40:0/498773228 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4ed9d640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41796/0 (socket says 172.21.15.40:41796) 2024-04-02T16:11:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fdc93fff640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 0x7fdc7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 -- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 msgr2=0x7f9d480c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 -- 172.21.15.40:0/498773228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d48037110 msgr2=0x7f9d4816cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d48037110 0x7f9d4816cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 -- 172.21.15.40:0/498773228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4800afc0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4ed9d640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fdc93fff640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 0x7fdc7806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc80006fd0 tx=0x7fdc80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f9d4dd9b640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c700 tx=0x7f9d4400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc42c011070 con 0x7fc428036c60 2024-04-02T16:11:38.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fc425ffb640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc42c086880 con 0x7fc428036c60 2024-04-02T16:11:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f24400071d0 con 0x7f2450072970 2024-04-02T16:11:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f243effd640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 0x7f243806a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.092+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2440086ff0 con 0x7f2450072970 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7f244dd9b640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 0x7f243806a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc005220 con 0x7fdee4035560 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc007030 con 0x7fdee4035560 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fc434f7d640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 0x7fc41406a840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc42001aa30 tx=0x7fc420008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7f244dd9b640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 0x7f243806a980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2450151b60 tx=0x7f2444008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc4280c0110 con 0x7fc428036c60 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdedc007290 con 0x7fdee4035560 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fdee1ffb640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 0x7fded006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.096+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdedc087020 con 0x7fdee4035560 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.100+0000 7fdef0819640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 0x7fded006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.100+0000 7fdef0819640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 0x7fded006a840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee80ad000 tx=0x7fdee8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.100+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fdc880c3030 con 0x7fdc9402c6f0 2024-04-02T16:11:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.100+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2440020020 con 0x7f2450072970 2024-04-02T16:11:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5684004240 con 0x7f568c141390 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44017070 con 0x7f9d4803b7f0 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d44005ac0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44007070 con 0x7f9d4803b7f0 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4800b2a0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d480c8e50 con 0x7f9d4803b7f0 2024-04-02T16:11:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d340052a0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f5673fff640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 0x7f567806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5684121ee0 con 0x7f568c141390 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.104+0000 7f568b7fe640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 0x7f567806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.108+0000 7f568b7fe640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 0x7f567806a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f567c0094a0 tx=0x7f567c0093f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.108+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fdedc020020 con 0x7fdee4035560 2024-04-02T16:11:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.116+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f568c0b30a0 con 0x7f568c141390 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.116+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9d4401a070 con 0x7f9d4803b7f0 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.116+0000 7f9d3effd640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 0x7f9d3006a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.116+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d44086280 con 0x7f9d4803b7f0 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.116+0000 7f9d4e59c640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 0x7f9d3006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.120+0000 7f9d4e59c640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 0x7f9d3006a890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d40007fd0 tx=0x7f9d40007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:38.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.128+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f9d44021020 con 0x7f9d4803b7f0 2024-04-02T16:11:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.396+0000 7f7d86cf6640 1 -- 172.21.15.40:0/4154905898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7d780cbff0 con 0x7f7d780cd420 2024-04-02T16:11:38.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.396+0000 7f7d7d7fa640 1 -- 172.21.15.40:0/4154905898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d80175310 con 0x7f7d780cd420 2024-04-02T16:11:38.402 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 msgr2=0x7f7d6406a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 0x7f7d6406a910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d740098f0 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2024-04-02T16:11:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 msgr2=0x7f7d780c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7d8006add0 tx=0x7f7d80069370 comp rx=0 tx=0).stop 2024-04-02T16:11:38.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 shutdown_connections 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d7803c230 0x7f7d780377c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7d64068450 0x7f7d6406a910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d780cd420 0x7f7d780c8840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 --2- 172.21.15.40:0/4154905898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d7800c360 0x7f7d781548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 >> 172.21.15.40:0/4154905898 conn(0x7f7d7800b5e0 msgr2=0x7f7d780069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 shutdown_connections 2024-04-02T16:11:38.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.400+0000 7f7d6affd640 1 -- 172.21.15.40:0/4154905898 wait complete. 2024-04-02T16:11:38.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.416+0000 7ff7f744d640 1 -- 172.21.15.40:0/570487484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff7d8005860 con 0x7ff7f0147450 2024-04-02T16:11:38.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.420+0000 7ff7e6ffd640 1 -- 172.21.15.40:0/570487484 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7e005b110 con 0x7ff7f0147450 2024-04-02T16:11:38.426 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2024-04-02T16:11:38.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-02T16:11:38.428 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.420+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 msgr2=0x7ff7d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.420+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 0x7ff7d406a870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0ae000 tx=0x7ff7ec0a8000 comp rx=0 tx=0).stop 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.420+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 msgr2=0x7ff7f01e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.420+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 0x7ff7f01e0270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000e930 tx=0x7ff7e000edf0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 shutdown_connections 2024-04-02T16:11:38.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff7f0144220 0x7ff7f01dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff7d40683b0 0x7ff7d406a870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7f0147450 0x7ff7f01e0270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 --2- 172.21.15.40:0/570487484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7f00ae6f0 0x7ff7f01e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 >> 172.21.15.40:0/570487484 conn(0x7ff7f00b8b80 msgr2=0x7ff7f013e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 shutdown_connections 2024-04-02T16:11:38.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.424+0000 7ff7e4ff9640 1 -- 172.21.15.40:0/570487484 wait complete. 2024-04-02T16:11:38.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.428+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdc9413f970 con 0x7fdc9402c6f0 2024-04-02T16:11:38.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.428+0000 7fdc917fa640 1 -- 172.21.15.40:0/4228219366 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc88016070 con 0x7fdc9402c6f0 2024-04-02T16:11:38.434 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc410005860 con 0x7fc428036c60 2024-04-02T16:11:38.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fc425ffb640 1 -- 172.21.15.40:0/3534185109 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc42c056e30 con 0x7fc428036c60 2024-04-02T16:11:38.436 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 msgr2=0x7fdc7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 0x7fdc7806a840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc80006fd0 tx=0x7fdc80008040 comp rx=0 tx=0).stop 2024-04-02T16:11:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 msgr2=0x7fdc94144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc880ae040 tx=0x7fdc88019160 comp rx=0 tx=0).stop 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 shutdown_connections 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc940b4a50 0x7fdc94140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdc78068380 0x7fdc7806a840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc940b7250 0x7fdc9413fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 --2- 172.21.15.40:0/4228219366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9402c6f0 0x7fdc94144720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 >> 172.21.15.40:0/4228219366 conn(0x7fdc940bb3e0 msgr2=0x7fdc940672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 shutdown_connections 2024-04-02T16:11:38.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.432+0000 7fdc9b083640 1 -- 172.21.15.40:0/4228219366 wait complete. 2024-04-02T16:11:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.436+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f24500baa00 con 0x7f2450072970 2024-04-02T16:11:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.436+0000 7f243effd640 1 -- 172.21.15.40:0/1162777069 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2450148050 con 0x7f2450072970 2024-04-02T16:11:38.443 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-02T16:11:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 msgr2=0x7fc41406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 0x7fc41406a840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc42001aa30 tx=0x7fc420008040 comp rx=0 tx=0).stop 2024-04-02T16:11:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 msgr2=0x7fc4280b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 0x7fc4280b0310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc42c000c10 tx=0x7fc42c00c8e0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 msgr2=0x7f243806a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 0x7f243806a980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2450151b60 tx=0x7f2444008040 comp rx=0 tx=0).stop 2024-04-02T16:11:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 shutdown_connections 2024-04-02T16:11:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 msgr2=0x7f2450143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f244000c530 tx=0x7f244000c9f0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc4280c9700 0x7fc4280c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 shutdown_connections 2024-04-02T16:11:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24500b44f0 0x7f2450150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc414068380 0x7fc41406a840 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f24380684c0 0x7f243806a980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24500b6080 0x7f245013a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 --2- 172.21.15.40:0/1162777069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2450072970 0x7f2450143850 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 >> 172.21.15.40:0/1162777069 conn(0x7f245006e4f0 msgr2=0x7f245006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428037c20 0x7fc4280c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 shutdown_connections 2024-04-02T16:11:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 --2- 172.21.15.40:0/3534185109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428036c60 0x7fc4280b0310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7f2454dbe640 1 -- 172.21.15.40:0/1162777069 wait complete. 2024-04-02T16:11:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.440+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 >> 172.21.15.40:0/3534185109 conn(0x7fc4280086c0 msgr2=0x7fc428005a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.444+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 shutdown_connections 2024-04-02T16:11:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.444+0000 7fc436780640 1 -- 172.21.15.40:0/3534185109 wait complete. 2024-04-02T16:11:38.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-04-02T16:11:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.444+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f568c04f870 con 0x7f568c141390 2024-04-02T16:11:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9d34005860 con 0x7f9d4803b7f0 2024-04-02T16:11:38.452 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f5673fff640 1 -- 172.21.15.40:0/4294215843 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56840c7020 con 0x7f568c141390 2024-04-02T16:11:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d3effd640 1 -- 172.21.15.40:0/498773228 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d480d09e0 con 0x7f9d4803b7f0 2024-04-02T16:11:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdee4029d00 con 0x7fdee4035560 2024-04-02T16:11:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7fdee1ffb640 1 -- 172.21.15.40:0/1335392428 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdedc057670 con 0x7fdee4035560 2024-04-02T16:11:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 msgr2=0x7f9d3006a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 0x7f9d3006a890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d40007fd0 tx=0x7f9d40007900 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 msgr2=0x7f9d4816d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c700 tx=0x7f9d4400cbc0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 shutdown_connections 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d480d15b0 0x7f9d480c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9d300683d0 0x7f9d3006a890 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d4803b7f0 0x7f9d4816d3e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 --2- 172.21.15.40:0/498773228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d48037110 0x7f9d4816cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 >> 172.21.15.40:0/498773228 conn(0x7f9d4803eca0 msgr2=0x7f9d4803c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 shutdown_connections 2024-04-02T16:11:38.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.448+0000 7f9d4f59e640 1 -- 172.21.15.40:0/498773228 wait complete. 2024-04-02T16:11:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 msgr2=0x7f567806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 0x7f567806a840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f567c0094a0 tx=0x7f567c0093f0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 msgr2=0x7f568c1e0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f56840b0040 tx=0x7f56840196d0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 shutdown_connections 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f568c0b6e90 0x7f568c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5678068380 0x7f567806a840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f568c141390 0x7f568c1e0490 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 --2- 172.21.15.40:0/4294215843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c073070 0x7f568c1e3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 >> 172.21.15.40:0/4294215843 conn(0x7f568c06ca90 msgr2=0x7f568c0b9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 shutdown_connections 2024-04-02T16:11:38.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7f5692200640 1 -- 172.21.15.40:0/4294215843 wait complete. 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 msgr2=0x7fded006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 0x7fded006a840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdee80ad000 tx=0x7fdee8004740 comp rx=0 tx=0).stop 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 msgr2=0x7fdee415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c500 tx=0x7fdedc00c9c0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 shutdown_connections 2024-04-02T16:11:38.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdee4035940 0x7fdee416a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fded0068380 0x7fded006a840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee40c8600 0x7fdee416cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 --2- 172.21.15.40:0/1335392428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdee4035560 0x7fdee415ce50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 >> 172.21.15.40:0/1335392428 conn(0x7fdee403abd0 msgr2=0x7fdee4006da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 shutdown_connections 2024-04-02T16:11:38.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:38.452+0000 7fdef201c640 1 -- 172.21.15.40:0/1335392428 wait complete. 2024-04-02T16:11:38.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-02T16:11:38.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-02T16:11:38.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-02T16:11:38.480 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:38.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-02T16:11:38.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-02T16:11:38.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-02T16:11:38.488 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:39.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-02T16:11:39.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-02T16:11:39.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T16:11:39.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-02T16:11:39.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 Processor -- start 2024-04-02T16:11:39.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 -- start start 2024-04-02T16:11:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f74d8072910 0x7f74d80af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d8071550 con 0x7f74d8148550 2024-04-02T16:11:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d8147a70 con 0x7f74d8141460 2024-04-02T16:11:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74dd289640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f74d8147d50 con 0x7f74d8072910 2024-04-02T16:11:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74d67fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f74d8148550 0x7f74d8070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50900/0 (socket says 172.21.15.40:50900) 2024-04-02T16:11:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74d67fc640 1 -- 172.21.15.40:0/453034225 learned_addr learned my addr 172.21.15.40:0/453034225 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/453034225 >> v2:172.21.15.204:3300/0 conn(0x7f74d8072910 0x7f74d80af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023089484 0 0) 0x7f74d8071550 con 0x7f74d8148550 2024-04-02T16:11:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.596+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74c0005740 con 0x7f74d8148550 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4206446639 0 0) 0x7f74c0005740 con 0x7f74d8148550 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 >> v2:172.21.15.204:3300/0 conn(0x7f74d8072910 msgr2=0x7f74d80af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 --2- 172.21.15.40:0/453034225 >> v2:172.21.15.204:3300/0 conn(0x7f74d8072910 0x7f74d80af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 >> v1:172.21.15.204:6789/0 conn(0x7f74d8141460 legacy=0x7f74d8141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8148030 con 0x7f74d8148550 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f74cc001680 con 0x7f74d8148550 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 >> v1:172.21.15.40:6789/0 conn(0x7f74d8148550 legacy=0x7f74d8070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c0005bd0 0x7f74c0005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74c0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74c00073e0 0x7f74c000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8147d50 con 0x7f74c0006a20 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74c000e330 con 0x7f74c0005bd0 2024-04-02T16:11:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f74c000e530 con 0x7f74c00073e0 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74c0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c0005bd0 0x7f74c0005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d77fe640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74c00073e0 0x7f74c000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 -- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74c00073e0 msgr2=0x7f74c000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74c00073e0 0x7f74c000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 -- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c0005bd0 msgr2=0x7f74c0005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c0005bd0 0x7f74c0005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 -- 172.21.15.40:0/453034225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c0005740 con 0x7f74c0006a20 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d77fe640 1 -- 172.21.15.40:0/453034225 reap_dead start 2024-04-02T16:11:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74c0005bd0 0x7f74c0005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d67fc640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74c0006ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c240 tx=0x7f74cc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d129aef5ca97dc26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc007d20 con 0x7f74c0006a20 2024-04-02T16:11:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc0a9020 con 0x7f74c0006a20 2024-04-02T16:11:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.600+0000 7f74d5ffb640 1 -- 172.21.15.40:0/453034225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0a5b60 con 0x7f74c0006a20 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 msgr2=0x7f74c0006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74c0006ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00c240 tx=0x7f74cc01cc60 comp rx=0 tx=0).stop 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/453034225 shutdown_connections 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 --2- 172.21.15.40:0/453034225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74c0006ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/453034225 >> 172.21.15.40:0/453034225 conn(0x7f74d80b7f50 msgr2=0x7f74d80b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/453034225 shutdown_connections 2024-04-02T16:11:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/453034225 wait complete. 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 Processor -- start 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- start start 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 0x7f74d80b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 0x7f74d80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80713d0 con 0x7f74c0006a20 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8071250 con 0x7f74d813fc80 2024-04-02T16:11:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f74d8141840 con 0x7f74d80b4cc0 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37342/0 (socket says 172.21.15.40:37342) 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 -- 172.21.15.40:0/2078823594 learned_addr learned my addr 172.21.15.40:0/2078823594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d67fc640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 0x7f74d80b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d77fe640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 0x7f74d80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 -- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 msgr2=0x7f74d80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 0x7f74d80b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 -- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 msgr2=0x7f74d80b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 0x7f74d80b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 -- 172.21.15.40:0/2078823594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8147600 con 0x7f74c0006a20 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d77fe640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 0x7f74d80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d67fc640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 0x7f74d80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74d6ffd640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74c8011540 tx=0x7f74c8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8004220 con 0x7f74c0006a20 2024-04-02T16:11:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c80043c0 con 0x7f74c0006a20 2024-04-02T16:11:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8015890 con 0x7f74c0006a20 2024-04-02T16:11:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/2078823594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d8145c90 con 0x7f74c0006a20 2024-04-02T16:11:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.604+0000 7f74dd289640 1 -- 172.21.15.40:0/2078823594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81459b0 con 0x7f74c0006a20 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b00052a0 con 0x7f74c0006a20 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f74c8015a30 con 0x7f74c0006a20 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74bf7fe640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 0x7f74c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74c8086f10 con 0x7f74c0006a20 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74d67fc640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 0x7f74c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.608+0000 7f74d67fc640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 0x7f74c406a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0aa000 tx=0x7f74d8148a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.612+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f74c8019030 con 0x7f74c0006a20 2024-04-02T16:11:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 Processor -- start 2024-04-02T16:11:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 -- start start 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15fc153070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc09f4c0 con 0x7f15fc09f0e0 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15fc0b0ff0 con 0x7f15fc070910 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f16034fc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc072f20 con 0x7f15fc0b0550 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601298640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f15fc09f0e0 0x7f15fc0713c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50902/0 (socket says 172.21.15.40:50902) 2024-04-02T16:11:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601298640 1 -- 172.21.15.40:0/1288900506 learned_addr learned my addr 172.21.15.40:0/1288900506 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601a99640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15fc153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3218301989 0 0) 0x7f15fc09f4c0 con 0x7f15fc09f0e0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15e8003530 con 0x7f15fc09f0e0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 578718261 0 0) 0x7f15e8003530 con 0x7f15fc09f0e0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 msgr2=0x7f15fc153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15fc153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 >> v1:172.21.15.204:6789/0 conn(0x7f15fc070910 legacy=0x7f15fc141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc073150 con 0x7f15fc09f0e0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f15f8001660 con 0x7f15fc09f0e0 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 >> v1:172.21.15.40:6789/0 conn(0x7f15fc09f0e0 legacy=0x7f15fc0713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e8003f70 0x7f15e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8004d50 0x7f15e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15e800bb10 con 0x7f15e8003f70 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15e800bcc0 con 0x7f15e8004d50 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f15e800bec0 con 0x7f15fc0b0550 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e8003f70 0x7f15e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601a99640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8004d50 0x7f15e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601298640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 -- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 msgr2=0x7f15e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15fc0b0550 0x7f15e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 -- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8004d50 msgr2=0x7f15e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8004d50 0x7f15e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1600a97640 1 -- 172.21.15.40:0/1288900506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e8003530 con 0x7f15e8003f70 2024-04-02T16:11:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.632+0000 7f1601a99640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15e8004d50 0x7f15e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f1600a97640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e8003f70 0x7f15e8004810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001780 tx=0x7f15ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709259f36e57569a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0013a0 con 0x7f15e8003f70 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ec0a9070 con 0x7f15e8003f70 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f15f3fff640 1 -- 172.21.15.40:0/1288900506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec003600 con 0x7f15e8003f70 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 -- 172.21.15.40:0/1288900506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e8003f70 msgr2=0x7f15e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 --2- 172.21.15.40:0/1288900506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15e8003f70 0x7f15e8004810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001780 tx=0x7f15ec01c420 comp rx=0 tx=0).stop 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f1601a99640 1 -- 172.21.15.40:0/1288900506 reap_dead start 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 -- 172.21.15.40:0/1288900506 shutdown_connections 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 -- 172.21.15.40:0/1288900506 >> 172.21.15.40:0/1288900506 conn(0x7f15fc0b7f50 msgr2=0x7f15fc0b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 -- 172.21.15.40:0/1288900506 shutdown_connections 2024-04-02T16:11:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.636+0000 7f16034fc640 1 -- 172.21.15.40:0/1288900506 wait complete. 2024-04-02T16:11:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 Processor -- start 2024-04-02T16:11:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- start start 2024-04-02T16:11:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 Processor -- start 2024-04-02T16:11:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 -- start start 2024-04-02T16:11:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8003f70 0x7f15fc148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15fc147770 0x7f15fc147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc0734c0 con 0x7f15fc146780 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc141d70 con 0x7f15fc147770 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f15fc141ef0 con 0x7f15e8003f70 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37368/0 (socket says 172.21.15.40:37368) 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 -- 172.21.15.40:0/2867845756 learned_addr learned my addr 172.21.15.40:0/2867845756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 Processor -- start 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 -- start start 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601298640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8003f70 0x7f15fc148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1600a97640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15fc147770 0x7f15fc147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 -- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8003f70 msgr2=0x7f15fc148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8003f70 0x7f15fc148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 -- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15fc147770 msgr2=0x7f15fc147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15fc147770 0x7f15fc147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 -- 172.21.15.40:0/2867845756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1480f0 con 0x7f15fc146780 2024-04-02T16:11:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f1601a99640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15fc14bdf0 tx=0x7f15f4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f80080b9cd0 0x7f80080ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f800814cee0 0x7f800814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f800814cee0 0x7f800814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f800814cee0 0x7f800814f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37384/0 (socket says 172.21.15.40:37384) 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 -- 172.21.15.40:0/511406316 learned_addr learned my addr 172.21.15.40:0/511406316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800802c840 con 0x7f800802c460 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4010da0 con 0x7f15fc146780 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f40039a0 con 0x7f15fc146780 2024-04-02T16:11:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4003d90 con 0x7f15fc146780 2024-04-02T16:11:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f80077fe640 1 --2- 172.21.15.40:0/511406316 >> v2:172.21.15.204:3300/0 conn(0x7f80080b9cd0 0x7f80080ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc147140 con 0x7f15fc146780 2024-04-02T16:11:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc149720 con 0x7f15fc146780 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbf00b4e70 0x7fdbf0148320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbf00b7a30 0x7fdbf00b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf0147880 con 0x7fdbf0141530 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf00b5250 con 0x7fdbf00b4e70 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf85f8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf0147b20 con 0x7fdbf00b7a30 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8008142210 con 0x7f800814cee0 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f80080b4cf0 con 0x7f80080b9cd0 2024-04-02T16:11:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f8006ffd640 1 -- 172.21.15.40:0/511406316 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1804250559 0 0) 0x7f800802c840 con 0x7f800802c460 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f8006ffd640 1 -- 172.21.15.40:0/511406316 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff0003530 con 0x7f800802c460 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbf00b7a30 0x7fdbf00b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbf00b7a30 0x7fdbf00b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55568/0 (socket says 172.21.15.40:55568) 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf6b95640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fdbf0141530 0x7fdbf014ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:41522/0 (socket says 172.21.15.40:41522) 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 -- 172.21.15.40:0/2569438636 learned_addr learned my addr 172.21.15.40:0/2569438636 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 -- 172.21.15.40:0/2569438636 >> v2:172.21.15.40:3300/0 conn(0x7fdbf00b4e70 msgr2=0x7fdbf0148320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T16:11:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc149920 con 0x7f15fc146780 2024-04-02T16:11:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 --2- 172.21.15.40:0/2569438636 >> v2:172.21.15.40:3300/0 conn(0x7fdbf00b4e70 0x7fdbf0148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 -- 172.21.15.40:0/2569438636 >> v1:172.21.15.204:6789/0 conn(0x7fdbf0141530 legacy=0x7fdbf014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7fdbf5b93640 1 -- 172.21.15.40:0/2569438636 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf0147e00 con 0x7fdbf00b7a30 2024-04-02T16:11:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f15f401e020 con 0x7f15fc146780 2024-04-02T16:11:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f15f1ffb640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 0x7f15e006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15f40870c0 con 0x7f15fc146780 2024-04-02T16:11:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f1601298640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 0x7f15e006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf5b93640 1 --2- 172.21.15.40:0/2569438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf00b7a30 0x7fdbf00b59f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00b76b0 tx=0x7fdbe001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce3cbc26b20e0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f1601298640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 0x7f15e006a8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15f80059a0 tx=0x7f15f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf5392640 1 -- 172.21.15.40:0/2569438636 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00b1070 con 0x7fdbf00b7a30 2024-04-02T16:11:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf5392640 1 -- 172.21.15.40:0/2569438636 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe00ac600 con 0x7fdbf00b7a30 2024-04-02T16:11:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf5392640 1 -- 172.21.15.40:0/2569438636 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00ac9d0 con 0x7fdbf00b7a30 2024-04-02T16:11:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 -- 172.21.15.40:0/511406316 >> v2:172.21.15.204:3300/0 conn(0x7f80080b9cd0 msgr2=0x7f80080ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 --2- 172.21.15.40:0/511406316 >> v2:172.21.15.204:3300/0 conn(0x7f80080b9cd0 0x7f80080ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 -- 172.21.15.40:0/511406316 >> v1:172.21.15.40:6789/0 conn(0x7f800802c460 legacy=0x7f8008141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.640+0000 7f800cdf4640 1 -- 172.21.15.40:0/511406316 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080aedf0 con 0x7f800814cee0 2024-04-02T16:11:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7f800cdf4640 1 --2- 172.21.15.40:0/511406316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800814cee0 0x7f800814f2d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8008073c30 tx=0x7f800001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526b83145728bc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2569438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf00b7a30 msgr2=0x7fdbf00b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2569438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf00b7a30 0x7fdbf00b59f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00b76b0 tx=0x7fdbe001c490 comp rx=0 tx=0).stop 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2569438636 shutdown_connections 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2569438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf00b7a30 0x7fdbf00b59f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2569438636 >> v2:172.21.15.40:3300/0 conn(0x7fdbf00b4e70 0x7fdbf0148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.644+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2569438636 >> 172.21.15.40:0/2569438636 conn(0x7fdbf00b9b30 msgr2=0x7fdbf00b9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2569438636 shutdown_connections 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2569438636 wait complete. 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 Processor -- start 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- start start 2024-04-02T16:11:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f8006ffd640 1 -- 172.21.15.40:0/511406316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5650 con 0x7f800814cee0 2024-04-02T16:11:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f8006ffd640 1 -- 172.21.15.40:0/511406316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80000a57f0 con 0x7f800814cee0 2024-04-02T16:11:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f8006ffd640 1 -- 172.21.15.40:0/511406316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5c00 con 0x7f800814cee0 2024-04-02T16:11:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbf00b7a30 0x7fdbf013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf0141530 0x7fdbf01e3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0155290 con 0x7fdbf00b4e70 2024-04-02T16:11:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0155110 con 0x7fdbf0141530 2024-04-02T16:11:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf85f8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf0155410 con 0x7fdbf00b7a30 2024-04-02T16:11:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f15fc149920 con 0x7f15fc146780 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37402/0 (socket says 172.21.15.40:37402) 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 -- 172.21.15.40:0/2021582366 learned_addr learned my addr 172.21.15.40:0/2021582366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf5b93640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbf00b7a30 0x7fdbf013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6b95640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf0141530 0x7fdbf01e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800814cee0 msgr2=0x7f800814f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 --2- 172.21.15.40:0/511406316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800814cee0 0x7f800814f2d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8008073c30 tx=0x7f800001c980 comp rx=0 tx=0).stop 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 shutdown_connections 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 --2- 172.21.15.40:0/511406316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800814cee0 0x7f800814f2d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 --2- 172.21.15.40:0/511406316 >> v2:172.21.15.204:3300/0 conn(0x7f80080b9cd0 0x7f80080ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 >> 172.21.15.40:0/511406316 conn(0x7f800809aa70 msgr2=0x7f80080b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 -- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbf00b7a30 msgr2=0x7fdbf013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbf00b7a30 0x7fdbf013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 -- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf0141530 msgr2=0x7fdbf01e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf0141530 0x7fdbf01e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 -- 172.21.15.40:0/2021582366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf0142430 con 0x7fdbf00b4e70 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.648+0000 7fdbf6394640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00b740 tx=0x7fdbec00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec019070 con 0x7fdbf00b4e70 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec00c7e0 con 0x7fdbf00b4e70 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf0144800 con 0x7fdbf00b4e70 2024-04-02T16:11:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00cbd0 con 0x7fdbf00b4e70 2024-04-02T16:11:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf0134f80 con 0x7fdbf00b4e70 2024-04-02T16:11:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdbec01c070 con 0x7fdbf00b4e70 2024-04-02T16:11:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 shutdown_connections 2024-04-02T16:11:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7f800e857640 1 -- 172.21.15.40:0/511406316 wait complete. 2024-04-02T16:11:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 0x7fdbd406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbec085e10 con 0x7fdbf00b4e70 2024-04-02T16:11:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.652+0000 7fdbf5b93640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 0x7fdbd406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 Processor -- start 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 -- start start 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 0x7f8008143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800814cee0 0x7f80081e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80080b4140 con 0x7f800802c460 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80080b3fc0 con 0x7f800814cee0 2024-04-02T16:11:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800e857640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80080b6cf0 con 0x7f80080b9cd0 2024-04-02T16:11:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7fdbf5b93640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 0x7fdbd406a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00ae000 tx=0x7fdbe00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf0068d00 con 0x7fdbf00b4e70 2024-04-02T16:11:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37418/0 (socket says 172.21.15.40:37418) 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 -- 172.21.15.40:0/3473988078 learned_addr learned my addr 172.21.15.40:0/3473988078 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f800cdf4640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800814cee0 0x7f80081e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f80077fe640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 0x7f8008143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 -- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 msgr2=0x7f8008143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 0x7f8008143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 -- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800814cee0 msgr2=0x7f80081e33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800814cee0 0x7f80081e33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 -- 172.21.15.40:0/3473988078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80081dff20 con 0x7f800802c460 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f80077fe640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 0x7f8008143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.656+0000 7f8007fff640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c430 tx=0x7f7ff800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8017070 con 0x7f800802c460 2024-04-02T16:11:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80081e0200 con 0x7f800802c460 2024-04-02T16:11:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fdbec055d70 con 0x7fdbf00b4e70 2024-04-02T16:11:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081406f0 con 0x7f800802c460 2024-04-02T16:11:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff80052d0 con 0x7f800802c460 2024-04-02T16:11:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8003800 con 0x7f800802c460 2024-04-02T16:11:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.664+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe80052a0 con 0x7f800802c460 2024-04-02T16:11:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.668+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7ff801a070 con 0x7f800802c460 2024-04-02T16:11:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.668+0000 7f8004ff9640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 0x7f7ff406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.668+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff8086460 con 0x7f800802c460 2024-04-02T16:11:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.668+0000 7f80077fe640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 0x7f7ff406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.668+0000 7f80077fe640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 0x7f7ff406a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80081dda40 tx=0x7f7ffc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.672+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7ff8005480 con 0x7f800802c460 2024-04-02T16:11:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.796+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f74b0005860 con 0x7f74c0006a20 2024-04-02T16:11:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.796+0000 7f74bf7fe640 1 -- 172.21.15.40:0/2078823594 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74c8057560 con 0x7f74c0006a20 2024-04-02T16:11:39.804 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:39.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 msgr2=0x7f74c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 0x7f74c406a840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0aa000 tx=0x7f74d8148a00 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 msgr2=0x7f74d813d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74c8011540 tx=0x7f74c8011a00 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 shutdown_connections 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74d80b4cc0 0x7f74d80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f74c4068380 0x7f74c406a840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74d813fc80 0x7f74d80b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 --2- 172.21.15.40:0/2078823594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c0006a20 0x7f74d813d7b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 >> 172.21.15.40:0/2078823594 conn(0x7f74d80b7f50 msgr2=0x7f74d8143560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 shutdown_connections 2024-04-02T16:11:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.800+0000 7f74bd7fa640 1 -- 172.21.15.40:0/2078823594 wait complete. 2024-04-02T16:11:39.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-02T16:11:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7fe8005860 con 0x7f800802c460 2024-04-02T16:11:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f8004ff9640 1 -- 172.21.15.40:0/3473988078 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ff80563c0 con 0x7f800802c460 2024-04-02T16:11:39.849 INFO:teuthology.orchestra.run.smithi040.stdout:55834574852 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 msgr2=0x7f7ff406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 0x7f7ff406a870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80081dda40 tx=0x7f7ffc008040 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 msgr2=0x7f80081e0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c430 tx=0x7f7ff800c8f0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 shutdown_connections 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80080b9cd0 0x7f8008143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7ff40683b0 0x7f7ff406a870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800814cee0 0x7f80081e33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 --2- 172.21.15.40:0/3473988078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800802c460 0x7f80081e0a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 >> 172.21.15.40:0/3473988078 conn(0x7f800809aa70 msgr2=0x7f80080672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 shutdown_connections 2024-04-02T16:11:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.844+0000 7f800e857640 1 -- 172.21.15.40:0/3473988078 wait complete. 2024-04-02T16:11:39.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-02T16:11:39.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.932+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f15fc068d00 con 0x7f15fc146780 2024-04-02T16:11:39.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.932+0000 7f15f1ffb640 1 -- 172.21.15.40:0/2867845756 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15f4025020 con 0x7f15fc146780 2024-04-02T16:11:39.938 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.932+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 msgr2=0x7f15e006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.932+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 0x7f15e006a8c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15f80059a0 tx=0x7f15f8009040 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.932+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 msgr2=0x7f15fc146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15fc14bdf0 tx=0x7f15f4010590 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 shutdown_connections 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f15e0068400 0x7f15e006a8c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15fc146780 0x7f15fc146c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15fc147770 0x7f15fc147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 --2- 172.21.15.40:0/2867845756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15e8003f70 0x7f15fc148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 >> 172.21.15.40:0/2867845756 conn(0x7f15fc0b7f50 msgr2=0x7f15fc13d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 shutdown_connections 2024-04-02T16:11:39.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.936+0000 7f16034fc640 1 -- 172.21.15.40:0/2867845756 wait complete. 2024-04-02T16:11:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.944+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdbf0145190 con 0x7fdbf00b4e70 2024-04-02T16:11:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.944+0000 7fdbe6ffd640 1 -- 172.21.15.40:0/2021582366 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbec05a070 con 0x7fdbf00b4e70 2024-04-02T16:11:39.951 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2024-04-02T16:11:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 msgr2=0x7fdbd406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 0x7fdbd406a870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00ae000 tx=0x7fdbe00a7000 comp rx=0 tx=0).stop 2024-04-02T16:11:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 msgr2=0x7fdbf01e0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00b740 tx=0x7fdbec00bc00 comp rx=0 tx=0).stop 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 shutdown_connections 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbf00b7a30 0x7fdbf013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdbd40683b0 0x7fdbd406a870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbf0141530 0x7fdbf01e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 --2- 172.21.15.40:0/2021582366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbf00b4e70 0x7fdbf01e0f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 >> 172.21.15.40:0/2021582366 conn(0x7fdbf00b9b30 msgr2=0x7fdbf0146e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 shutdown_connections 2024-04-02T16:11:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:39.948+0000 7fdbf85f8640 1 -- 172.21.15.40:0/2021582366 wait complete. 2024-04-02T16:11:39.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-04-02T16:11:39.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-02T16:11:40.822 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-02T16:11:40.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-02T16:11:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 Processor -- start 2024-04-02T16:11:40.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 -- start start 2024-04-02T16:11:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b7802c6f0 0x7f3b7802cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b780b4a50 0x7f3b780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b780b5a50 con 0x7f3b780b7250 2024-04-02T16:11:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3b780b31f0 con 0x7f3b7802c6f0 2024-04-02T16:11:40.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.948+0000 7f3b7ef04640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b78145c10 con 0x7f3b780b4a50 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7cca0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b780b7250 0x7f3b78145500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50924/0 (socket says 172.21.15.40:50924) 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b7802c6f0 0x7f3b7802cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7cca0640 1 -- 172.21.15.40:0/1655329582 learned_addr learned my addr 172.21.15.40:0/1655329582 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7d4a1640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b780b4a50 0x7f3b780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b777fe640 1 -- 172.21.15.40:0/1655329582 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374954279 0 0) 0x7f3b780b5a50 con 0x7f3b780b7250 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b777fe640 1 -- 172.21.15.40:0/1655329582 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b64003530 con 0x7f3b780b7250 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 -- 172.21.15.40:0/1655329582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b780b4a50 msgr2=0x7f3b780b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b780b4a50 0x7f3b780b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 -- 172.21.15.40:0/1655329582 >> v1:172.21.15.40:6789/0 conn(0x7f3b780b7250 legacy=0x7f3b78145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:40.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 -- 172.21.15.40:0/1655329582 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b780b7630 con 0x7f3b7802c6f0 2024-04-02T16:11:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b77fff640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b7802c6f0 0x7f3b7802cb80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b600097a0 tx=0x7f3b6001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748540695e7c5ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b777fe640 1 -- 172.21.15.40:0/1655329582 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b600b1070 con 0x7f3b7802c6f0 2024-04-02T16:11:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b777fe640 1 -- 172.21.15.40:0/1655329582 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b600ac620 con 0x7f3b7802c6f0 2024-04-02T16:11:40.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b777fe640 1 -- 172.21.15.40:0/1655329582 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b600ac9f0 con 0x7f3b7802c6f0 2024-04-02T16:11:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 -- 172.21.15.40:0/1655329582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b7802c6f0 msgr2=0x7f3b7802cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b7802c6f0 0x7f3b7802cb80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3b600097a0 tx=0x7f3b6001c3c0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 -- 172.21.15.40:0/1655329582 shutdown_connections 2024-04-02T16:11:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b780b4a50 0x7f3b780b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/1655329582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b7802c6f0 0x7f3b7802cb80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 -- 172.21.15.40:0/1655329582 >> 172.21.15.40:0/1655329582 conn(0x7f3b780bb3e0 msgr2=0x7f3b780bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 -- 172.21.15.40:0/1655329582 shutdown_connections 2024-04-02T16:11:40.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.952+0000 7f3b7ef04640 1 -- 172.21.15.40:0/1655329582 wait complete. 2024-04-02T16:11:40.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-02T16:11:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 Processor -- start 2024-04-02T16:11:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- start start 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 0x7f3b7813f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780b7250 0x7f3b781e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b780b64c0 con 0x7f3b780b4a50 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78152620 con 0x7f3b780b7250 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7813c460 con 0x7f3b7802c6f0 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37436/0 (socket says 172.21.15.40:37436) 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 -- 172.21.15.40:0/2884599613 learned_addr learned my addr 172.21.15.40:0/2884599613 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7cca0640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 0x7f3b7813f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 -- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 msgr2=0x7f3b7813f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 0x7f3b7813f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 -- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780b7250 msgr2=0x7f3b781e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:40.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780b7250 0x7f3b781e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 -- 172.21.15.40:0/2884599613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7813dcd0 con 0x7f3b780b4a50 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7cca0640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 0x7f3b7813f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b77fff640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b60019ff0 tx=0x7f3b6001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b600b1050 con 0x7f3b780b4a50 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b600bd070 con 0x7f3b780b4a50 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b600b8690 con 0x7f3b780b4a50 2024-04-02T16:11:40.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7813dfb0 con 0x7f3b780b4a50 2024-04-02T16:11:40.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b7813c940 con 0x7f3b780b4a50 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78068d00 con 0x7f3b780b4a50 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.956+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3b600b8830 con 0x7f3b780b4a50 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.960+0000 7f3b757fa640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 0x7f3b5806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.960+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b601210f0 con 0x7f3b780b4a50 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.960+0000 7f3b7cca0640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 0x7f3b5806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:40.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.960+0000 7f3b7cca0640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 0x7f3b5806a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005890 tx=0x7f3b70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:40.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:40.964+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3b60016070 con 0x7f3b780b4a50 2024-04-02T16:11:40.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-02T16:11:41.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 Processor -- start 2024-04-02T16:11:41.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 -- start start 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f915409f0e0 0x7f9154072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9154072ec0 0x7f9154147e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540affd0 0x7f9154148380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9154141fb0 con 0x7f91540affd0 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9154072040 con 0x7f915409f0e0 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f915b909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f91541477f0 con 0x7f9154072ec0 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540affd0 0x7f9154148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f91540affd0 0x7f9154148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37446/0 (socket says 172.21.15.40:37446) 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 -- 172.21.15.40:0/10424142 learned_addr learned my addr 172.21.15.40:0/10424142 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 -- 172.21.15.40:0/10424142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9154072ec0 msgr2=0x7f9154147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 --2- 172.21.15.40:0/10424142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9154072ec0 0x7f9154147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 -- 172.21.15.40:0/10424142 >> v2:172.21.15.204:3300/0 conn(0x7f915409f0e0 msgr2=0x7f9154072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 --2- 172.21.15.40:0/10424142 >> v2:172.21.15.204:3300/0 conn(0x7f915409f0e0 0x7f9154072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.000+0000 7f9159ea6640 1 -- 172.21.15.40:0/10424142 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154070a60 con 0x7f91540affd0 2024-04-02T16:11:41.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f9159ea6640 1 --2- 172.21.15.40:0/10424142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540affd0 0x7f9154148380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9154070c30 tx=0x7f914c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1ad6435b68e5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f914bfff640 1 -- 172.21.15.40:0/10424142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c0a5640 con 0x7f91540affd0 2024-04-02T16:11:41.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f914bfff640 1 -- 172.21.15.40:0/10424142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914c0a57e0 con 0x7f91540affd0 2024-04-02T16:11:41.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f914bfff640 1 -- 172.21.15.40:0/10424142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c0a5bb0 con 0x7f91540affd0 2024-04-02T16:11:41.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- 172.21.15.40:0/10424142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540affd0 msgr2=0x7f9154148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 --2- 172.21.15.40:0/10424142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540affd0 0x7f9154148380 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9154070c30 tx=0x7f914c01c8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- 172.21.15.40:0/10424142 shutdown_connections 2024-04-02T16:11:41.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 --2- 172.21.15.40:0/10424142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91540affd0 0x7f9154148380 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 --2- 172.21.15.40:0/10424142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9154072ec0 0x7f9154147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 --2- 172.21.15.40:0/10424142 >> v2:172.21.15.204:3300/0 conn(0x7f915409f0e0 0x7f9154072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- 172.21.15.40:0/10424142 >> 172.21.15.40:0/10424142 conn(0x7f91540b7d10 msgr2=0x7f91540b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- 172.21.15.40:0/10424142 shutdown_connections 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- 172.21.15.40:0/10424142 wait complete. 2024-04-02T16:11:41.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 Processor -- start 2024-04-02T16:11:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.004+0000 7f915b909640 1 -- start start 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f915409f0e0 0x7f915413b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91540affd0 0x7f91541e3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9154073450 con 0x7f9154072ec0 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f915409fee0 con 0x7f915409f0e0 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f915414b570 con 0x7f91540affd0 2024-04-02T16:11:41.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37460/0 (socket says 172.21.15.40:37460) 2024-04-02T16:11:41.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 -- 172.21.15.40:0/1328140387 learned_addr learned my addr 172.21.15.40:0/1328140387 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9159ea6640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91540affd0 0x7f91541e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9158ea4640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f915409f0e0 0x7f915413b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 -- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91540affd0 msgr2=0x7f91541e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91540affd0 0x7f91541e3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 -- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f915409f0e0 msgr2=0x7f915413b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f915409f0e0 0x7f915413b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 -- 172.21.15.40:0/1328140387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154143f40 con 0x7f9154072ec0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f91596a5640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f915000b580 tx=0x7f915000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150014070 con 0x7f9154072ec0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9150010890 con 0x7f9154072ec0 2024-04-02T16:11:41.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9154144fa0 con 0x7f9154072ec0 2024-04-02T16:11:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150010c80 con 0x7f9154072ec0 2024-04-02T16:11:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9154143c60 con 0x7f9154072ec0 2024-04-02T16:11:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9154068d00 con 0x7f9154072ec0 2024-04-02T16:11:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9150017070 con 0x7f9154072ec0 2024-04-02T16:11:41.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 0x7f913806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9150086770 con 0x7f9154072ec0 2024-04-02T16:11:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.008+0000 7f9158ea4640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 0x7f913806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.012+0000 7f9158ea4640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 0x7f913806a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9144006fd0 tx=0x7f9144008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.016+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f9150056650 con 0x7f9154072ec0 2024-04-02T16:11:41.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 Processor -- start 2024-04-02T16:11:41.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 -- start start 2024-04-02T16:11:41.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc138150e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1380b5610 con 0x7fc1380b6070 2024-04-02T16:11:41.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1380b6450 con 0x7fc1380b4060 2024-04-02T16:11:41.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13e2c7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc138073330 con 0x7fc1380b47c0 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1380b6070 0x7fc1380b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50940/0 (socket says 172.21.15.40:50940) 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 -- 172.21.15.40:0/3849224103 learned_addr learned my addr 172.21.15.40:0/3849224103 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13c864640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc138150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2273659611 0 0) 0x7fc1380b5610 con 0x7fc1380b6070 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc120003530 con 0x7fc1380b6070 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1521840214 0 0) 0x7fc120003530 con 0x7fc1380b6070 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 msgr2=0x7fc138150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc138150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 >> v1:172.21.15.204:6789/0 conn(0x7fc1380b4060 legacy=0x7fc138145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc138072d70 con 0x7fc1380b6070 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fc1280016a0 con 0x7fc1380b6070 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 >> v1:172.21.15.40:6789/0 conn(0x7fc1380b6070 legacy=0x7fc1380b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc120004810 0x7fc120004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc120005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc138073330 con 0x7fc1200039f0 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc12000bb60 con 0x7fc120004810 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc12000bd60 con 0x7fc1380b47c0 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13c864640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc120005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc1377fe640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc120004810 0x7fc120004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 -- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 msgr2=0x7fc120005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc120005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 -- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc120004810 msgr2=0x7fc120004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc120004810 0x7fc120004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc137fff640 1 -- 172.21.15.40:0/3849224103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc138072d70 con 0x7fc1200039f0 2024-04-02T16:11:41.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.064+0000 7fc13c864640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1380b47c0 0x7fc120005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc137fff640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c340 tx=0x7fc12801cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245455c62845c5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128007e00 con 0x7fc1200039f0 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1280a9030 con 0x7fc1200039f0 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc136ffd640 1 -- 172.21.15.40:0/3849224103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128005d00 con 0x7fc1200039f0 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 -- 172.21.15.40:0/3849224103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1200039f0 msgr2=0x7fc120003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/3849224103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1200039f0 0x7fc120003e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c340 tx=0x7fc12801cd60 comp rx=0 tx=0).stop 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13c864640 1 -- 172.21.15.40:0/3849224103 reap_dead start 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 -- 172.21.15.40:0/3849224103 shutdown_connections 2024-04-02T16:11:41.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 -- 172.21.15.40:0/3849224103 >> 172.21.15.40:0/3849224103 conn(0x7fc13806e600 msgr2=0x7fc13806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 -- 172.21.15.40:0/3849224103 shutdown_connections 2024-04-02T16:11:41.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.068+0000 7fc13e2c7640 1 -- 172.21.15.40:0/3849224103 wait complete. 2024-04-02T16:11:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 Processor -- start 2024-04-02T16:11:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- start start 2024-04-02T16:11:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc13813eef0 0x7fc138071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 0x7fc138131e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc138151810 con 0x7fc13813f2d0 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc138151690 con 0x7fc13813df60 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc138151990 con 0x7fc13813eef0 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37478/0 (socket says 172.21.15.40:37478) 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 -- 172.21.15.40:0/1129107132 learned_addr learned my addr 172.21.15.40:0/1129107132 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13c864640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 0x7fc138131e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc137fff640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc13813eef0 0x7fc138071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 -- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc13813eef0 msgr2=0x7fc138071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc13813eef0 0x7fc138071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 -- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 msgr2=0x7fc138131e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 0x7fc138131e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 -- 172.21.15.40:0/1129107132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13813e490 con 0x7fc13813f2d0 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13c864640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 0x7fc138131e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc1377fe640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e4b0 tx=0x7fc12c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0052e0 con 0x7fc13813f2d0 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12c018070 con 0x7fc13813f2d0 2024-04-02T16:11:41.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13813e770 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c003820 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13814e980 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.072+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc12c0039c0 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc138068d00 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fc134ff9640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc12c08a170 con 0x7fc13813f2d0 2024-04-02T16:11:41.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fc137fff640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 Processor -- start 2024-04-02T16:11:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 -- start start 2024-04-02T16:11:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fc137fff640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 0x7fc12406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c340 tx=0x7fc1280b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb30c13e280 0x7fb30c13e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb30c147bf0 con 0x7fb30c1486d0 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb30c1479c0 con 0x7fb30c13e280 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb311e3d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c147ed0 con 0x7fb30c0b3e00 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:41908/0 (socket says 172.21.15.40:41908) 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 -- 172.21.15.40:0/3311597488 learned_addr learned my addr 172.21.15.40:0/3311597488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 -- 172.21.15.40:0/3311597488 >> v2:172.21.15.40:3300/0 conn(0x7fb30c13e280 msgr2=0x7fb30c13e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 --2- 172.21.15.40:0/3311597488 >> v2:172.21.15.40:3300/0 conn(0x7fb30c13e280 0x7fb30c13e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 -- 172.21.15.40:0/3311597488 >> v1:172.21.15.204:6789/0 conn(0x7fb30c1486d0 legacy=0x7fb30c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.076+0000 7fb30b7fe640 1 -- 172.21.15.40:0/3311597488 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1481b0 con 0x7fb30c0b3e00 2024-04-02T16:11:41.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30b7fe640 1 --2- 172.21.15.40:0/3311597488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009870 tx=0x7fb2fc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120e174fee01014c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30a7fc640 1 -- 172.21.15.40:0/3311597488 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0b0070 con 0x7fb30c0b3e00 2024-04-02T16:11:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30a7fc640 1 -- 172.21.15.40:0/3311597488 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc0ab600 con 0x7fb30c0b3e00 2024-04-02T16:11:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30a7fc640 1 -- 172.21.15.40:0/3311597488 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0ab9d0 con 0x7fb30c0b3e00 2024-04-02T16:11:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc12c005480 con 0x7fc13813f2d0 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- 172.21.15.40:0/3311597488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 msgr2=0x7fb30c0b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- 172.21.15.40:0/3311597488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009870 tx=0x7fb2fc01c4d0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- 172.21.15.40:0/3311597488 shutdown_connections 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- 172.21.15.40:0/3311597488 >> v2:172.21.15.40:3300/0 conn(0x7fb30c13e280 0x7fb30c13e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- 172.21.15.40:0/3311597488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c0b2df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- 172.21.15.40:0/3311597488 >> 172.21.15.40:0/3311597488 conn(0x7fb30c0b91a0 msgr2=0x7fb30c0b95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- 172.21.15.40:0/3311597488 shutdown_connections 2024-04-02T16:11:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- 172.21.15.40:0/3311597488 wait complete. 2024-04-02T16:11:41.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 Processor -- start 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- start start 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c1e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c1486d0 0x7fb30c130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1516f0 con 0x7fb30c13e280 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c151570 con 0x7fb30c1486d0 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb311e3d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c151870 con 0x7fb30c0b3e00 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37502/0 (socket says 172.21.15.40:37502) 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.080+0000 7fb30affd640 1 -- 172.21.15.40:0/835998298 learned_addr learned my addr 172.21.15.40:0/835998298 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30b7fe640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c1e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30bfff640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c1486d0 0x7fb30c130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 -- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 msgr2=0x7fb30c1e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c1e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 -- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c1486d0 msgr2=0x7fb30c130e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c1486d0 0x7fb30c130e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 -- 172.21.15.40:0/835998298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1e22b0 con 0x7fb30c13e280 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30affd640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c420 tx=0x7fb30000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000052d0 con 0x7fb30c13e280 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb300005470 con 0x7fb30c13e280 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb311e3d640 1 -- 172.21.15.40:0/835998298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1e1570 con 0x7fb30c13e280 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300007260 con 0x7fb30c13e280 2024-04-02T16:11:41.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb311e3d640 1 -- 172.21.15.40:0/835998298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c145fd0 con 0x7fb30c13e280 2024-04-02T16:11:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e40052a0 con 0x7fb30c13e280 2024-04-02T16:11:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb300007400 con 0x7fb30c13e280 2024-04-02T16:11:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 0x7fb2f806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb300054670 con 0x7fb30c13e280 2024-04-02T16:11:41.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.084+0000 7fb30b7fe640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 0x7fb2f806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.088+0000 7fb30b7fe640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 0x7fb2f806a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0ad000 tx=0x7fb2fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.088+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb300056cd0 con 0x7fb30c13e280 2024-04-02T16:11:41.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.240+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc138144e80 con 0x7fc13813f2d0 2024-04-02T16:11:41.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.240+0000 7fc134ff9640 1 -- 172.21.15.40:0/1129107132 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc12c0562d0 con 0x7fc13813f2d0 2024-04-02T16:11:41.246 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 msgr2=0x7fc12406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 0x7fc12406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c340 tx=0x7fc1280b0040 comp rx=0 tx=0).stop 2024-04-02T16:11:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 msgr2=0x7fc138072050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00e4b0 tx=0x7fc12c00e970 comp rx=0 tx=0).stop 2024-04-02T16:11:41.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 shutdown_connections 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc1240683b0 0x7fc12406a870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc13813df60 0x7fc138131e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc13813f2d0 0x7fc138072050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 --2- 172.21.15.40:0/1129107132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc13813eef0 0x7fc138071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 >> 172.21.15.40:0/1129107132 conn(0x7fc13806e600 msgr2=0x7fc1380b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 shutdown_connections 2024-04-02T16:11:41.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.244+0000 7fc13e2c7640 1 -- 172.21.15.40:0/1129107132 wait complete. 2024-04-02T16:11:41.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.248+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb2e4005860 con 0x7fb30c13e280 2024-04-02T16:11:41.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.248+0000 7fb2f3fff640 1 -- 172.21.15.40:0/835998298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3000565d0 con 0x7fb30c13e280 2024-04-02T16:11:41.254 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 msgr2=0x7fb2f806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 0x7fb2f806a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0ad000 tx=0x7fb2fc008040 comp rx=0 tx=0).stop 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 msgr2=0x7fb30c145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c420 tx=0x7fb30000c8e0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 shutdown_connections 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb2f8068380 0x7fb2f806a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb30c1486d0 0x7fb30c130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb30c13e280 0x7fb30c145a90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 --2- 172.21.15.40:0/835998298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb30c0b3e00 0x7fb30c1e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 >> 172.21.15.40:0/835998298 conn(0x7fb30c0b91a0 msgr2=0x7fb30c1446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 shutdown_connections 2024-04-02T16:11:41.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.252+0000 7fb2f1ffb640 1 -- 172.21.15.40:0/835998298 wait complete. 2024-04-02T16:11:41.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-02T16:11:41.265 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:41.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-02T16:11:41.272 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:41.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.276+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3b7814f990 con 0x7f3b780b4a50 2024-04-02T16:11:41.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.276+0000 7f3b757fa640 1 -- 172.21.15.40:0/2884599613 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b600f16a0 con 0x7f3b780b4a50 2024-04-02T16:11:41.281 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2024-04-02T16:11:41.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 msgr2=0x7f3b5806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 0x7f3b5806a840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005890 tx=0x7f3b70009040 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 msgr2=0x7f3b7813fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3b60019ff0 tx=0x7f3b6001ac00 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 shutdown_connections 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3b58068380 0x7f3b5806a840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780b7250 0x7f3b781e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b780b4a50 0x7f3b7813fc50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 --2- 172.21.15.40:0/2884599613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b7802c6f0 0x7f3b7813f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 >> 172.21.15.40:0/2884599613 conn(0x7f3b780bb3e0 msgr2=0x7f3b780b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 shutdown_connections 2024-04-02T16:11:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.280+0000 7f3b7ef04640 1 -- 172.21.15.40:0/2884599613 wait complete. 2024-04-02T16:11:41.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-02T16:11:41.305 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:41.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f91541470b0 con 0x7f9154072ec0 2024-04-02T16:11:41.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f9149ffb640 1 -- 172.21.15.40:0/1328140387 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9150010e20 con 0x7f9154072ec0 2024-04-02T16:11:41.325 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2024-04-02T16:11:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 msgr2=0x7f913806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 0x7f913806a840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9144006fd0 tx=0x7f9144008040 comp rx=0 tx=0).stop 2024-04-02T16:11:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 msgr2=0x7f915413aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f915000b580 tx=0x7f915000ba40 comp rx=0 tx=0).stop 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 shutdown_connections 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91540affd0 0x7f91541e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9138068380 0x7f913806a840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f915409f0e0 0x7f915413b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 --2- 172.21.15.40:0/1328140387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9154072ec0 0x7f915413aeb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 >> 172.21.15.40:0/1328140387 conn(0x7f91540b7d10 msgr2=0x7f91540b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 shutdown_connections 2024-04-02T16:11:41.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.320+0000 7f915b909640 1 -- 172.21.15.40:0/1328140387 wait complete. 2024-04-02T16:11:41.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-02T16:11:41.342 DEBUG:teuthology.parallel:result is None 2024-04-02T16:11:41.342 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-02T16:11:41.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:11:41.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.492+0000 7f659c817640 1 Processor -- start 2024-04-02T16:11:41.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.492+0000 7f659c817640 1 -- start start 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659c817640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f659806c8a0 0x7f65980af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659c817640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65980b3800 0x7f65980b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659c817640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65980b4640 con 0x7f65980b41d0 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659c817640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f65980b2b50 con 0x7f659806c8a0 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659c817640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f65980b0300 con 0x7f65980b3800 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f659806c8a0 0x7f65980af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f659806c8a0 0x7f65980af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37510/0 (socket says 172.21.15.40:37510) 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 -- 172.21.15.40:0/2992279100 learned_addr learned my addr 172.21.15.40:0/2992279100 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f6595d9b640 1 --2- 172.21.15.40:0/2992279100 >> v2:172.21.15.204:3300/0 conn(0x7f65980b3800 0x7f65980b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 -- 172.21.15.40:0/2992279100 >> v2:172.21.15.204:3300/0 conn(0x7f65980b3800 msgr2=0x7f65980b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 --2- 172.21.15.40:0/2992279100 >> v2:172.21.15.204:3300/0 conn(0x7f65980b3800 0x7f65980b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 -- 172.21.15.40:0/2992279100 >> v1:172.21.15.204:6789/0 conn(0x7f65980b41d0 legacy=0x7f6598147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 -- 172.21.15.40:0/2992279100 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65980ae830 con 0x7f659806c8a0 2024-04-02T16:11:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659659c640 1 --2- 172.21.15.40:0/2992279100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65980af150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f659806e9c0 tx=0x7f658801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367024851f1b5895 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659559a640 1 -- 172.21.15.40:0/2992279100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880a55f0 con 0x7f659806c8a0 2024-04-02T16:11:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659559a640 1 -- 172.21.15.40:0/2992279100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65880a5790 con 0x7f659806c8a0 2024-04-02T16:11:41.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.496+0000 7f659559a640 1 -- 172.21.15.40:0/2992279100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880a5b60 con 0x7f659806c8a0 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/2992279100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 msgr2=0x7f65980af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- 172.21.15.40:0/2992279100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65980af150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f659806e9c0 tx=0x7f658801c920 comp rx=0 tx=0).stop 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/2992279100 shutdown_connections 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- 172.21.15.40:0/2992279100 >> v2:172.21.15.204:3300/0 conn(0x7f65980b3800 0x7f65980b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- 172.21.15.40:0/2992279100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65980af150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/2992279100 >> 172.21.15.40:0/2992279100 conn(0x7f65980b8fd0 msgr2=0x7f65980b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/2992279100 shutdown_connections 2024-04-02T16:11:41.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/2992279100 wait complete. 2024-04-02T16:11:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 Processor -- start 2024-04-02T16:11:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- start start 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65980b3800 0x7f65980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 0x7f6598070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65980af690 con 0x7f659806c8a0 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65980b0540 con 0x7f65980b3800 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f65980b3130 con 0x7f65980b41d0 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37524/0 (socket says 172.21.15.40:37524) 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 -- 172.21.15.40:0/303451275 learned_addr learned my addr 172.21.15.40:0/303451275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6596d9d640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 0x7f6598070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6595d9b640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65980b3800 0x7f65980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 -- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 msgr2=0x7f6598070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 0x7f6598070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 -- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65980b3800 msgr2=0x7f65980706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65980b3800 0x7f65980706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 -- 172.21.15.40:0/303451275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659814fbb0 con 0x7f659806c8a0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6596d9d640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 0x7f6598070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659659c640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65880af040 tx=0x7f65880196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880bc020 con 0x7f659806c8a0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65880b8880 con 0x7f659806c8a0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588009510 con 0x7f659806c8a0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659814fe20 con 0x7f659806c8a0 2024-04-02T16:11:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659806feb0 con 0x7f659806c8a0 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.500+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65980685f0 con 0x7f659806c8a0 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.504+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f65880bf020 con 0x7f659806c8a0 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.504+0000 7f6586ffd640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 0x7f657806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.504+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6588121db0 con 0x7f659806c8a0 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.504+0000 7f6595d9b640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 0x7f657806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.504+0000 7f6595d9b640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 0x7f657806a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65981dd5a0 tx=0x7f658c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.508+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f65880c6030 con 0x7f659806c8a0 2024-04-02T16:11:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.660+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f659806bf70 con 0x7f6578068380 2024-04-02T16:11:41.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f6586ffd640 1 -- 172.21.15.40:0/303451275 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251884 (secure 0 0 0) 0x7f659806bf70 con 0x7f6578068380 2024-04-02T16:11:41.683 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:41.686 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:11:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 msgr2=0x7f657806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 0x7f657806a840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65981dd5a0 tx=0x7f658c008040 comp rx=0 tx=0).stop 2024-04-02T16:11:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 msgr2=0x7f65981dfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65880af040 tx=0x7f65880196c0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 shutdown_connections 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65980b41d0 0x7f6598070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6578068380 0x7f657806a840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65980b3800 0x7f65980706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 --2- 172.21.15.40:0/303451275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659806c8a0 0x7f65981dfc40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 >> 172.21.15.40:0/303451275 conn(0x7f65980b8fd0 msgr2=0x7f65980b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 shutdown_connections 2024-04-02T16:11:41.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.680+0000 7f659c817640 1 -- 172.21.15.40:0/303451275 wait complete. 2024-04-02T16:11:41.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-02T16:11:39.850482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003058"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497949+0000","last_change":"2024-04-02T16:11:32.176361+0000","last_active":"2024-04-02T16:11:33.497949+0000","last_peered":"2024-04-02T16:11:33.497949+0000","last_clean":"2024-04-02T16:11:33.497949+0000","last_became_active":"2024-04-02T16:11:32.176117+0000","last_became_peered":"2024-04-02T16:11:32.176117+0000","last_unstale":"2024-04-02T16:11:33.497949+0000","last_undegraded":"2024-04-02T16:11:33.497949+0000","last_fullsized":"2024-04-02T16:11:33.497949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:03.664122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156872+0000","last_change":"2024-04-02T16:11:32.165208+0000","last_active":"2024-04-02T16:11:33.156872+0000","last_peered":"2024-04-02T16:11:33.156872+0000","last_clean":"2024-04-02T16:11:33.156872+0000","last_became_active":"2024-04-02T16:11:32.164869+0000","last_became_peered":"2024-04-02T16:11:32.164869+0000","last_unstale":"2024-04-02T16:11:33.156872+0000","last_undegraded":"2024-04-02T16:11:33.156872+0000","last_fullsized":"2024-04-02T16:11:33.156872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892407+0000","last_change":"2024-04-02T16:11:32.146435+0000","last_active":"2024-04-02T16:11:34.892407+0000","last_peered":"2024-04-02T16:11:34.892407+0000","last_clean":"2024-04-02T16:11:34.892407+0000","last_became_active":"2024-04-02T16:11:32.145704+0000","last_became_peered":"2024-04-02T16:11:32.145704+0000","last_unstale":"2024-04-02T16:11:34.892407+0000","last_undegraded":"2024-04-02T16:11:34.892407+0000","last_fullsized":"2024-04-02T16:11:34.892407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158980+0000","last_change":"2024-04-02T16:11:32.172527+0000","last_active":"2024-04-02T16:11:33.158980+0000","last_peered":"2024-04-02T16:11:33.158980+0000","last_clean":"2024-04-02T16:11:33.158980+0000","last_became_active":"2024-04-02T16:11:32.170021+0000","last_became_peered":"2024-04-02T16:11:32.170021+0000","last_unstale":"2024-04-02T16:11:33.158980+0000","last_undegraded":"2024-04-02T16:11:33.158980+0000","last_fullsized":"2024-04-02T16:11:33.158980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497274+0000","last_change":"2024-04-02T16:11:32.171344+0000","last_active":"2024-04-02T16:11:33.497274+0000","last_peered":"2024-04-02T16:11:33.497274+0000","last_clean":"2024-04-02T16:11:33.497274+0000","last_became_active":"2024-04-02T16:11:32.170838+0000","last_became_peered":"2024-04-02T16:11:32.170838+0000","last_unstale":"2024-04-02T16:11:33.497274+0000","last_undegraded":"2024-04-02T16:11:33.497274+0000","last_fullsized":"2024-04-02T16:11:33.497274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158169+0000","last_change":"2024-04-02T16:11:32.160886+0000","last_active":"2024-04-02T16:11:33.158169+0000","last_peered":"2024-04-02T16:11:33.158169+0000","last_clean":"2024-04-02T16:11:33.158169+0000","last_became_active":"2024-04-02T16:11:32.159998+0000","last_became_peered":"2024-04-02T16:11:32.159998+0000","last_unstale":"2024-04-02T16:11:33.158169+0000","last_undegraded":"2024-04-02T16:11:33.158169+0000","last_fullsized":"2024-04-02T16:11:33.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496848+0000","last_change":"2024-04-02T16:11:32.172231+0000","last_active":"2024-04-02T16:11:33.496848+0000","last_peered":"2024-04-02T16:11:33.496848+0000","last_clean":"2024-04-02T16:11:33.496848+0000","last_became_active":"2024-04-02T16:11:32.171852+0000","last_became_peered":"2024-04-02T16:11:32.171852+0000","last_unstale":"2024-04-02T16:11:33.496848+0000","last_undegraded":"2024-04-02T16:11:33.496848+0000","last_fullsized":"2024-04-02T16:11:33.496848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889767+0000","last_change":"2024-04-02T16:11:32.171342+0000","last_active":"2024-04-02T16:11:34.889767+0000","last_peered":"2024-04-02T16:11:34.889767+0000","last_clean":"2024-04-02T16:11:34.889767+0000","last_became_active":"2024-04-02T16:11:32.171027+0000","last_became_peered":"2024-04-02T16:11:32.171027+0000","last_unstale":"2024-04-02T16:11:34.889767+0000","last_undegraded":"2024-04-02T16:11:34.889767+0000","last_fullsized":"2024-04-02T16:11:34.889767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:34:40.739755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498216+0000","last_change":"2024-04-02T16:11:32.171461+0000","last_active":"2024-04-02T16:11:33.498216+0000","last_peered":"2024-04-02T16:11:33.498216+0000","last_clean":"2024-04-02T16:11:33.498216+0000","last_became_active":"2024-04-02T16:11:32.170971+0000","last_became_peered":"2024-04-02T16:11:32.170971+0000","last_unstale":"2024-04-02T16:11:33.498216+0000","last_undegraded":"2024-04-02T16:11:33.498216+0000","last_fullsized":"2024-04-02T16:11:33.498216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:25.000722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161307+0000","last_change":"2024-04-02T16:11:32.164006+0000","last_active":"2024-04-02T16:11:34.161307+0000","last_peered":"2024-04-02T16:11:34.161307+0000","last_clean":"2024-04-02T16:11:34.161307+0000","last_became_active":"2024-04-02T16:11:32.163712+0000","last_became_peered":"2024-04-02T16:11:32.163712+0000","last_unstale":"2024-04-02T16:11:34.161307+0000","last_undegraded":"2024-04-02T16:11:34.161307+0000","last_fullsized":"2024-04-02T16:11:34.161307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160963+0000","last_change":"2024-04-02T16:11:32.162643+0000","last_active":"2024-04-02T16:11:34.160963+0000","last_peered":"2024-04-02T16:11:34.160963+0000","last_clean":"2024-04-02T16:11:34.160963+0000","last_became_active":"2024-04-02T16:11:32.162329+0000","last_became_peered":"2024-04-02T16:11:32.162329+0000","last_unstale":"2024-04-02T16:11:34.160963+0000","last_undegraded":"2024-04-02T16:11:34.160963+0000","last_fullsized":"2024-04-02T16:11:34.160963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158378+0000","last_change":"2024-04-02T16:11:32.150507+0000","last_active":"2024-04-02T16:11:33.158378+0000","last_peered":"2024-04-02T16:11:33.158378+0000","last_clean":"2024-04-02T16:11:33.158378+0000","last_became_active":"2024-04-02T16:11:32.149817+0000","last_became_peered":"2024-04-02T16:11:32.149817+0000","last_unstale":"2024-04-02T16:11:33.158378+0000","last_undegraded":"2024-04-02T16:11:33.158378+0000","last_fullsized":"2024-04-02T16:11:33.158378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890701+0000","last_change":"2024-04-02T16:11:32.171214+0000","last_active":"2024-04-02T16:11:34.890701+0000","last_peered":"2024-04-02T16:11:34.890701+0000","last_clean":"2024-04-02T16:11:34.890701+0000","last_became_active":"2024-04-02T16:11:32.170528+0000","last_became_peered":"2024-04-02T16:11:32.170528+0000","last_unstale":"2024-04-02T16:11:34.890701+0000","last_undegraded":"2024-04-02T16:11:34.890701+0000","last_fullsized":"2024-04-02T16:11:34.890701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891411+0000","last_change":"2024-04-02T16:11:32.148238+0000","last_active":"2024-04-02T16:11:34.891411+0000","last_peered":"2024-04-02T16:11:34.891411+0000","last_clean":"2024-04-02T16:11:34.891411+0000","last_became_active":"2024-04-02T16:11:32.147348+0000","last_became_peered":"2024-04-02T16:11:32.147348+0000","last_unstale":"2024-04-02T16:11:34.891411+0000","last_undegraded":"2024-04-02T16:11:34.891411+0000","last_fullsized":"2024-04-02T16:11:34.891411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:34:40.739755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156485+0000","last_change":"2024-04-02T16:11:32.174912+0000","last_active":"2024-04-02T16:11:33.156485+0000","last_peered":"2024-04-02T16:11:33.156485+0000","last_clean":"2024-04-02T16:11:33.156485+0000","last_became_active":"2024-04-02T16:11:32.174581+0000","last_became_peered":"2024-04-02T16:11:32.174581+0000","last_unstale":"2024-04-02T16:11:33.156485+0000","last_undegraded":"2024-04-02T16:11:33.156485+0000","last_fullsized":"2024-04-02T16:11:33.156485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156962+0000","last_change":"2024-04-02T16:11:32.160602+0000","last_active":"2024-04-02T16:11:33.156962+0000","last_peered":"2024-04-02T16:11:33.156962+0000","last_clean":"2024-04-02T16:11:33.156962+0000","last_became_active":"2024-04-02T16:11:32.160289+0000","last_became_peered":"2024-04-02T16:11:32.160289+0000","last_unstale":"2024-04-02T16:11:33.156962+0000","last_undegraded":"2024-04-02T16:11:33.156962+0000","last_fullsized":"2024-04-02T16:11:33.156962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159486+0000","last_change":"2024-04-02T16:11:32.159580+0000","last_active":"2024-04-02T16:11:33.159486+0000","last_peered":"2024-04-02T16:11:33.159486+0000","last_clean":"2024-04-02T16:11:33.159486+0000","last_became_active":"2024-04-02T16:11:32.159248+0000","last_became_peered":"2024-04-02T16:11:32.159248+0000","last_unstale":"2024-04-02T16:11:33.159486+0000","last_undegraded":"2024-04-02T16:11:33.159486+0000","last_fullsized":"2024-04-02T16:11:33.159486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891860+0000","last_change":"2024-04-02T16:11:32.152173+0000","last_active":"2024-04-02T16:11:34.891860+0000","last_peered":"2024-04-02T16:11:34.891860+0000","last_clean":"2024-04-02T16:11:34.891860+0000","last_became_active":"2024-04-02T16:11:32.150901+0000","last_became_peered":"2024-04-02T16:11:32.150901+0000","last_unstale":"2024-04-02T16:11:34.891860+0000","last_undegraded":"2024-04-02T16:11:34.891860+0000","last_fullsized":"2024-04-02T16:11:34.891860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497849+0000","last_change":"2024-04-02T16:11:32.174590+0000","last_active":"2024-04-02T16:11:33.497849+0000","last_peered":"2024-04-02T16:11:33.497849+0000","last_clean":"2024-04-02T16:11:33.497849+0000","last_became_active":"2024-04-02T16:11:32.174269+0000","last_became_peered":"2024-04-02T16:11:32.174269+0000","last_unstale":"2024-04-02T16:11:33.497849+0000","last_undegraded":"2024-04-02T16:11:33.497849+0000","last_fullsized":"2024-04-02T16:11:33.497849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160975+0000","last_change":"2024-04-02T16:11:32.174103+0000","last_active":"2024-04-02T16:11:34.160975+0000","last_peered":"2024-04-02T16:11:34.160975+0000","last_clean":"2024-04-02T16:11:34.160975+0000","last_became_active":"2024-04-02T16:11:32.173844+0000","last_became_peered":"2024-04-02T16:11:32.173844+0000","last_unstale":"2024-04-02T16:11:34.160975+0000","last_undegraded":"2024-04-02T16:11:34.160975+0000","last_fullsized":"2024-04-02T16:11:34.160975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893151+0000","last_change":"2024-04-02T16:11:32.155248+0000","last_active":"2024-04-02T16:11:34.893151+0000","last_peered":"2024-04-02T16:11:34.893151+0000","last_clean":"2024-04-02T16:11:34.893151+0000","last_became_active":"2024-04-02T16:11:32.154493+0000","last_became_peered":"2024-04-02T16:11:32.154493+0000","last_unstale":"2024-04-02T16:11:34.893151+0000","last_undegraded":"2024-04-02T16:11:34.893151+0000","last_fullsized":"2024-04-02T16:11:34.893151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158188+0000","last_change":"2024-04-02T16:11:32.163843+0000","last_active":"2024-04-02T16:11:33.158188+0000","last_peered":"2024-04-02T16:11:33.158188+0000","last_clean":"2024-04-02T16:11:33.158188+0000","last_became_active":"2024-04-02T16:11:32.163493+0000","last_became_peered":"2024-04-02T16:11:32.163493+0000","last_unstale":"2024-04-02T16:11:33.158188+0000","last_undegraded":"2024-04-02T16:11:33.158188+0000","last_fullsized":"2024-04-02T16:11:33.158188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890013+0000","last_change":"2024-04-02T16:11:32.160644+0000","last_active":"2024-04-02T16:11:34.890013+0000","last_peered":"2024-04-02T16:11:34.890013+0000","last_clean":"2024-04-02T16:11:34.890013+0000","last_became_active":"2024-04-02T16:11:32.160265+0000","last_became_peered":"2024-04-02T16:11:32.160265+0000","last_unstale":"2024-04-02T16:11:34.890013+0000","last_undegraded":"2024-04-02T16:11:34.890013+0000","last_fullsized":"2024-04-02T16:11:34.890013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157349+0000","last_change":"2024-04-02T16:11:32.169639+0000","last_active":"2024-04-02T16:11:33.157349+0000","last_peered":"2024-04-02T16:11:33.157349+0000","last_clean":"2024-04-02T16:11:33.157349+0000","last_became_active":"2024-04-02T16:11:32.166290+0000","last_became_peered":"2024-04-02T16:11:32.166290+0000","last_unstale":"2024-04-02T16:11:33.157349+0000","last_undegraded":"2024-04-02T16:11:33.157349+0000","last_fullsized":"2024-04-02T16:11:33.157349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891192+0000","last_change":"2024-04-02T16:11:32.172667+0000","last_active":"2024-04-02T16:11:34.891192+0000","last_peered":"2024-04-02T16:11:34.891192+0000","last_clean":"2024-04-02T16:11:34.891192+0000","last_became_active":"2024-04-02T16:11:32.171185+0000","last_became_peered":"2024-04-02T16:11:32.171185+0000","last_unstale":"2024-04-02T16:11:34.891192+0000","last_undegraded":"2024-04-02T16:11:34.891192+0000","last_fullsized":"2024-04-02T16:11:34.891192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161390+0000","last_change":"2024-04-02T16:11:32.174173+0000","last_active":"2024-04-02T16:11:34.161390+0000","last_peered":"2024-04-02T16:11:34.161390+0000","last_clean":"2024-04-02T16:11:34.161390+0000","last_became_active":"2024-04-02T16:11:32.173799+0000","last_became_peered":"2024-04-02T16:11:32.173799+0000","last_unstale":"2024-04-02T16:11:34.161390+0000","last_undegraded":"2024-04-02T16:11:34.161390+0000","last_fullsized":"2024-04-02T16:11:34.161390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156692+0000","last_change":"2024-04-02T16:11:32.169323+0000","last_active":"2024-04-02T16:11:33.156692+0000","last_peered":"2024-04-02T16:11:33.156692+0000","last_clean":"2024-04-02T16:11:33.156692+0000","last_became_active":"2024-04-02T16:11:32.169028+0000","last_became_peered":"2024-04-02T16:11:32.169028+0000","last_unstale":"2024-04-02T16:11:33.156692+0000","last_undegraded":"2024-04-02T16:11:33.156692+0000","last_fullsized":"2024-04-02T16:11:33.156692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891206+0000","last_change":"2024-04-02T16:11:32.177772+0000","last_active":"2024-04-02T16:11:34.891206+0000","last_peered":"2024-04-02T16:11:34.891206+0000","last_clean":"2024-04-02T16:11:34.891206+0000","last_became_active":"2024-04-02T16:11:32.177123+0000","last_became_peered":"2024-04-02T16:11:32.177123+0000","last_unstale":"2024-04-02T16:11:34.891206+0000","last_undegraded":"2024-04-02T16:11:34.891206+0000","last_fullsized":"2024-04-02T16:11:34.891206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893485+0000","last_change":"2024-04-02T16:11:32.168117+0000","last_active":"2024-04-02T16:11:34.893485+0000","last_peered":"2024-04-02T16:11:34.893485+0000","last_clean":"2024-04-02T16:11:34.893485+0000","last_became_active":"2024-04-02T16:11:32.167687+0000","last_became_peered":"2024-04-02T16:11:32.167687+0000","last_unstale":"2024-04-02T16:11:34.893485+0000","last_undegraded":"2024-04-02T16:11:34.893485+0000","last_fullsized":"2024-04-02T16:11:34.893485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158588+0000","last_change":"2024-04-02T16:11:32.168501+0000","last_active":"2024-04-02T16:11:33.158588+0000","last_peered":"2024-04-02T16:11:33.158588+0000","last_clean":"2024-04-02T16:11:33.158588+0000","last_became_active":"2024-04-02T16:11:32.168164+0000","last_became_peered":"2024-04-02T16:11:32.168164+0000","last_unstale":"2024-04-02T16:11:33.158588+0000","last_undegraded":"2024-04-02T16:11:33.158588+0000","last_fullsized":"2024-04-02T16:11:33.158588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889779+0000","last_change":"2024-04-02T16:11:32.152124+0000","last_active":"2024-04-02T16:11:34.889779+0000","last_peered":"2024-04-02T16:11:34.889779+0000","last_clean":"2024-04-02T16:11:34.889779+0000","last_became_active":"2024-04-02T16:11:32.150859+0000","last_became_peered":"2024-04-02T16:11:32.150859+0000","last_unstale":"2024-04-02T16:11:34.889779+0000","last_undegraded":"2024-04-02T16:11:34.889779+0000","last_fullsized":"2024-04-02T16:11:34.889779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160495+0000","last_change":"2024-04-02T16:11:32.166672+0000","last_active":"2024-04-02T16:11:34.160495+0000","last_peered":"2024-04-02T16:11:34.160495+0000","last_clean":"2024-04-02T16:11:34.160495+0000","last_became_active":"2024-04-02T16:11:32.165841+0000","last_became_peered":"2024-04-02T16:11:32.165841+0000","last_unstale":"2024-04-02T16:11:34.160495+0000","last_undegraded":"2024-04-02T16:11:34.160495+0000","last_fullsized":"2024-04-02T16:11:34.160495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157143+0000","last_change":"2024-04-02T16:11:32.148450+0000","last_active":"2024-04-02T16:11:33.157143+0000","last_peered":"2024-04-02T16:11:33.157143+0000","last_clean":"2024-04-02T16:11:33.157143+0000","last_became_active":"2024-04-02T16:11:32.147722+0000","last_became_peered":"2024-04-02T16:11:32.147722+0000","last_unstale":"2024-04-02T16:11:33.157143+0000","last_undegraded":"2024-04-02T16:11:33.157143+0000","last_fullsized":"2024-04-02T16:11:33.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890986+0000","last_change":"2024-04-02T16:11:31.123742+0000","last_active":"2024-04-02T16:11:34.890986+0000","last_peered":"2024-04-02T16:11:34.890986+0000","last_clean":"2024-04-02T16:11:34.890986+0000","last_became_active":"2024-04-02T16:11:31.123404+0000","last_became_peered":"2024-04-02T16:11:31.123404+0000","last_unstale":"2024-04-02T16:11:34.890986+0000","last_undegraded":"2024-04-02T16:11:34.890986+0000","last_fullsized":"2024-04-02T16:11:34.890986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893703+0000","last_change":"2024-04-02T16:11:32.140773+0000","last_active":"2024-04-02T16:11:34.893703+0000","last_peered":"2024-04-02T16:11:34.893703+0000","last_clean":"2024-04-02T16:11:34.893703+0000","last_became_active":"2024-04-02T16:11:32.140008+0000","last_became_peered":"2024-04-02T16:11:32.140008+0000","last_unstale":"2024-04-02T16:11:34.893703+0000","last_undegraded":"2024-04-02T16:11:34.893703+0000","last_fullsized":"2024-04-02T16:11:34.893703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160916+0000","last_change":"2024-04-02T16:11:31.131833+0000","last_active":"2024-04-02T16:11:34.160916+0000","last_peered":"2024-04-02T16:11:34.160916+0000","last_clean":"2024-04-02T16:11:34.160916+0000","last_became_active":"2024-04-02T16:11:31.130705+0000","last_became_peered":"2024-04-02T16:11:31.130705+0000","last_unstale":"2024-04-02T16:11:34.160916+0000","last_undegraded":"2024-04-02T16:11:34.160916+0000","last_fullsized":"2024-04-02T16:11:34.160916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890052+0000","last_change":"2024-04-02T16:11:31.130860+0000","last_active":"2024-04-02T16:11:34.890052+0000","last_peered":"2024-04-02T16:11:34.890052+0000","last_clean":"2024-04-02T16:11:34.890052+0000","last_became_active":"2024-04-02T16:11:31.130391+0000","last_became_peered":"2024-04-02T16:11:31.130391+0000","last_unstale":"2024-04-02T16:11:34.890052+0000","last_undegraded":"2024-04-02T16:11:34.890052+0000","last_fullsized":"2024-04-02T16:11:34.890052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890211+0000","last_change":"2024-04-02T16:11:32.166977+0000","last_active":"2024-04-02T16:11:34.890211+0000","last_peered":"2024-04-02T16:11:34.890211+0000","last_clean":"2024-04-02T16:11:34.890211+0000","last_became_active":"2024-04-02T16:11:32.166690+0000","last_became_peered":"2024-04-02T16:11:32.166690+0000","last_unstale":"2024-04-02T16:11:34.890211+0000","last_undegraded":"2024-04-02T16:11:34.890211+0000","last_fullsized":"2024-04-02T16:11:34.890211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157628+0000","last_change":"2024-04-02T16:11:31.128418+0000","last_active":"2024-04-02T16:11:33.157628+0000","last_peered":"2024-04-02T16:11:33.157628+0000","last_clean":"2024-04-02T16:11:33.157628+0000","last_became_active":"2024-04-02T16:11:31.126329+0000","last_became_peered":"2024-04-02T16:11:31.126329+0000","last_unstale":"2024-04-02T16:11:33.157628+0000","last_undegraded":"2024-04-02T16:11:33.157628+0000","last_fullsized":"2024-04-02T16:11:33.157628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156477+0000","last_change":"2024-04-02T16:11:32.164197+0000","last_active":"2024-04-02T16:11:33.156477+0000","last_peered":"2024-04-02T16:11:33.156477+0000","last_clean":"2024-04-02T16:11:33.156477+0000","last_became_active":"2024-04-02T16:11:32.163440+0000","last_became_peered":"2024-04-02T16:11:32.163440+0000","last_unstale":"2024-04-02T16:11:33.156477+0000","last_undegraded":"2024-04-02T16:11:33.156477+0000","last_fullsized":"2024-04-02T16:11:33.156477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161113+0000","last_change":"2024-04-02T16:11:31.137185+0000","last_active":"2024-04-02T16:11:34.161113+0000","last_peered":"2024-04-02T16:11:34.161113+0000","last_clean":"2024-04-02T16:11:34.161113+0000","last_became_active":"2024-04-02T16:11:31.136386+0000","last_became_peered":"2024-04-02T16:11:31.136386+0000","last_unstale":"2024-04-02T16:11:34.161113+0000","last_undegraded":"2024-04-02T16:11:34.161113+0000","last_fullsized":"2024-04-02T16:11:34.161113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158908+0000","last_change":"2024-04-02T16:11:32.159269+0000","last_active":"2024-04-02T16:11:33.158908+0000","last_peered":"2024-04-02T16:11:33.158908+0000","last_clean":"2024-04-02T16:11:33.158908+0000","last_became_active":"2024-04-02T16:11:32.158906+0000","last_became_peered":"2024-04-02T16:11:32.158906+0000","last_unstale":"2024-04-02T16:11:33.158908+0000","last_undegraded":"2024-04-02T16:11:33.158908+0000","last_fullsized":"2024-04-02T16:11:33.158908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159157+0000","last_change":"2024-04-02T16:11:31.133206+0000","last_active":"2024-04-02T16:11:33.159157+0000","last_peered":"2024-04-02T16:11:33.159157+0000","last_clean":"2024-04-02T16:11:33.159157+0000","last_became_active":"2024-04-02T16:11:31.132012+0000","last_became_peered":"2024-04-02T16:11:31.132012+0000","last_unstale":"2024-04-02T16:11:33.159157+0000","last_undegraded":"2024-04-02T16:11:33.159157+0000","last_fullsized":"2024-04-02T16:11:33.159157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893414+0000","last_change":"2024-04-02T16:11:32.153558+0000","last_active":"2024-04-02T16:11:34.893414+0000","last_peered":"2024-04-02T16:11:34.893414+0000","last_clean":"2024-04-02T16:11:34.893414+0000","last_became_active":"2024-04-02T16:11:32.152697+0000","last_became_peered":"2024-04-02T16:11:32.152697+0000","last_unstale":"2024-04-02T16:11:34.893414+0000","last_undegraded":"2024-04-02T16:11:34.893414+0000","last_fullsized":"2024-04-02T16:11:34.893414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156779+0000","last_change":"2024-04-02T16:11:31.134656+0000","last_active":"2024-04-02T16:11:33.156779+0000","last_peered":"2024-04-02T16:11:33.156779+0000","last_clean":"2024-04-02T16:11:33.156779+0000","last_became_active":"2024-04-02T16:11:31.134191+0000","last_became_peered":"2024-04-02T16:11:31.134191+0000","last_unstale":"2024-04-02T16:11:33.156779+0000","last_undegraded":"2024-04-02T16:11:33.156779+0000","last_fullsized":"2024-04-02T16:11:33.156779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158486+0000","last_change":"2024-04-02T16:11:32.154312+0000","last_active":"2024-04-02T16:11:33.158486+0000","last_peered":"2024-04-02T16:11:33.158486+0000","last_clean":"2024-04-02T16:11:33.158486+0000","last_became_active":"2024-04-02T16:11:32.153984+0000","last_became_peered":"2024-04-02T16:11:32.153984+0000","last_unstale":"2024-04-02T16:11:33.158486+0000","last_undegraded":"2024-04-02T16:11:33.158486+0000","last_fullsized":"2024-04-02T16:11:33.158486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161022+0000","last_change":"2024-04-02T16:11:31.128529+0000","last_active":"2024-04-02T16:11:34.161022+0000","last_peered":"2024-04-02T16:11:34.161022+0000","last_clean":"2024-04-02T16:11:34.161022+0000","last_became_active":"2024-04-02T16:11:31.127760+0000","last_became_peered":"2024-04-02T16:11:31.127760+0000","last_unstale":"2024-04-02T16:11:34.161022+0000","last_undegraded":"2024-04-02T16:11:34.161022+0000","last_fullsized":"2024-04-02T16:11:34.161022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497089+0000","last_change":"2024-04-02T16:11:32.169840+0000","last_active":"2024-04-02T16:11:33.497089+0000","last_peered":"2024-04-02T16:11:33.497089+0000","last_clean":"2024-04-02T16:11:33.497089+0000","last_became_active":"2024-04-02T16:11:32.167352+0000","last_became_peered":"2024-04-02T16:11:32.167352+0000","last_unstale":"2024-04-02T16:11:33.497089+0000","last_undegraded":"2024-04-02T16:11:33.497089+0000","last_fullsized":"2024-04-02T16:11:33.497089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893832+0000","last_change":"2024-04-02T16:11:31.135077+0000","last_active":"2024-04-02T16:11:34.893832+0000","last_peered":"2024-04-02T16:11:34.893832+0000","last_clean":"2024-04-02T16:11:34.893832+0000","last_became_active":"2024-04-02T16:11:31.134519+0000","last_became_peered":"2024-04-02T16:11:31.134519+0000","last_unstale":"2024-04-02T16:11:34.893832+0000","last_undegraded":"2024-04-02T16:11:34.893832+0000","last_fullsized":"2024-04-02T16:11:34.893832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891799+0000","last_change":"2024-04-02T16:11:32.155398+0000","last_active":"2024-04-02T16:11:34.891799+0000","last_peered":"2024-04-02T16:11:34.891799+0000","last_clean":"2024-04-02T16:11:34.891799+0000","last_became_active":"2024-04-02T16:11:32.154518+0000","last_became_peered":"2024-04-02T16:11:32.154518+0000","last_unstale":"2024-04-02T16:11:34.891799+0000","last_undegraded":"2024-04-02T16:11:34.891799+0000","last_fullsized":"2024-04-02T16:11:34.891799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160202+0000","last_change":"2024-04-02T16:11:31.125926+0000","last_active":"2024-04-02T16:11:33.160202+0000","last_peered":"2024-04-02T16:11:33.160202+0000","last_clean":"2024-04-02T16:11:33.160202+0000","last_became_active":"2024-04-02T16:11:31.124109+0000","last_became_peered":"2024-04-02T16:11:31.124109+0000","last_unstale":"2024-04-02T16:11:33.160202+0000","last_undegraded":"2024-04-02T16:11:33.160202+0000","last_fullsized":"2024-04-02T16:11:33.160202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497615+0000","last_change":"2024-04-02T16:11:32.171153+0000","last_active":"2024-04-02T16:11:33.497615+0000","last_peered":"2024-04-02T16:11:33.497615+0000","last_clean":"2024-04-02T16:11:33.497615+0000","last_became_active":"2024-04-02T16:11:32.170399+0000","last_became_peered":"2024-04-02T16:11:32.170399+0000","last_unstale":"2024-04-02T16:11:33.497615+0000","last_undegraded":"2024-04-02T16:11:33.497615+0000","last_fullsized":"2024-04-02T16:11:33.497615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160399+0000","last_change":"2024-04-02T16:11:31.131680+0000","last_active":"2024-04-02T16:11:34.160399+0000","last_peered":"2024-04-02T16:11:34.160399+0000","last_clean":"2024-04-02T16:11:34.160399+0000","last_became_active":"2024-04-02T16:11:31.129432+0000","last_became_peered":"2024-04-02T16:11:31.129432+0000","last_unstale":"2024-04-02T16:11:34.160399+0000","last_undegraded":"2024-04-02T16:11:34.160399+0000","last_fullsized":"2024-04-02T16:11:34.160399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160538+0000","last_change":"2024-04-02T16:11:32.167516+0000","last_active":"2024-04-02T16:11:34.160538+0000","last_peered":"2024-04-02T16:11:34.160538+0000","last_clean":"2024-04-02T16:11:34.160538+0000","last_became_active":"2024-04-02T16:11:32.167201+0000","last_became_peered":"2024-04-02T16:11:32.167201+0000","last_unstale":"2024-04-02T16:11:34.160538+0000","last_undegraded":"2024-04-02T16:11:34.160538+0000","last_fullsized":"2024-04-02T16:11:34.160538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889403+0000","last_change":"2024-04-02T16:11:31.117868+0000","last_active":"2024-04-02T16:11:34.889403+0000","last_peered":"2024-04-02T16:11:34.889403+0000","last_clean":"2024-04-02T16:11:34.889403+0000","last_became_active":"2024-04-02T16:11:31.117437+0000","last_became_peered":"2024-04-02T16:11:31.117437+0000","last_unstale":"2024-04-02T16:11:34.889403+0000","last_undegraded":"2024-04-02T16:11:34.889403+0000","last_fullsized":"2024-04-02T16:11:34.889403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497088+0000","last_change":"2024-04-02T16:11:32.178110+0000","last_active":"2024-04-02T16:11:33.497088+0000","last_peered":"2024-04-02T16:11:33.497088+0000","last_clean":"2024-04-02T16:11:33.497088+0000","last_became_active":"2024-04-02T16:11:32.177756+0000","last_became_peered":"2024-04-02T16:11:32.177756+0000","last_unstale":"2024-04-02T16:11:33.497088+0000","last_undegraded":"2024-04-02T16:11:33.497088+0000","last_fullsized":"2024-04-02T16:11:33.497088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159238+0000","last_change":"2024-04-02T16:11:31.130795+0000","last_active":"2024-04-02T16:11:33.159238+0000","last_peered":"2024-04-02T16:11:33.159238+0000","last_clean":"2024-04-02T16:11:33.159238+0000","last_became_active":"2024-04-02T16:11:31.129907+0000","last_became_peered":"2024-04-02T16:11:31.129907+0000","last_unstale":"2024-04-02T16:11:33.159238+0000","last_undegraded":"2024-04-02T16:11:33.159238+0000","last_fullsized":"2024-04-02T16:11:33.159238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:27:40.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497755+0000","last_change":"2024-04-02T16:11:32.173519+0000","last_active":"2024-04-02T16:11:33.497755+0000","last_peered":"2024-04-02T16:11:33.497755+0000","last_clean":"2024-04-02T16:11:33.497755+0000","last_became_active":"2024-04-02T16:11:32.173159+0000","last_became_peered":"2024-04-02T16:11:32.173159+0000","last_unstale":"2024-04-02T16:11:33.497755+0000","last_undegraded":"2024-04-02T16:11:33.497755+0000","last_fullsized":"2024-04-02T16:11:33.497755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892119+0000","last_change":"2024-04-02T16:11:31.113477+0000","last_active":"2024-04-02T16:11:34.892119+0000","last_peered":"2024-04-02T16:11:34.892119+0000","last_clean":"2024-04-02T16:11:34.892119+0000","last_became_active":"2024-04-02T16:11:31.112346+0000","last_became_peered":"2024-04-02T16:11:31.112346+0000","last_unstale":"2024-04-02T16:11:34.892119+0000","last_undegraded":"2024-04-02T16:11:34.892119+0000","last_fullsized":"2024-04-02T16:11:34.892119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498157+0000","last_change":"2024-04-02T16:11:32.173602+0000","last_active":"2024-04-02T16:11:33.498157+0000","last_peered":"2024-04-02T16:11:33.498157+0000","last_clean":"2024-04-02T16:11:33.498157+0000","last_became_active":"2024-04-02T16:11:32.173294+0000","last_became_peered":"2024-04-02T16:11:32.173294+0000","last_unstale":"2024-04-02T16:11:33.498157+0000","last_undegraded":"2024-04-02T16:11:33.498157+0000","last_fullsized":"2024-04-02T16:11:33.498157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:50.854339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156803+0000","last_change":"2024-04-02T16:11:31.134514+0000","last_active":"2024-04-02T16:11:33.156803+0000","last_peered":"2024-04-02T16:11:33.156803+0000","last_clean":"2024-04-02T16:11:33.156803+0000","last_became_active":"2024-04-02T16:11:31.134123+0000","last_became_peered":"2024-04-02T16:11:31.134123+0000","last_unstale":"2024-04-02T16:11:33.156803+0000","last_undegraded":"2024-04-02T16:11:33.156803+0000","last_fullsized":"2024-04-02T16:11:33.156803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159396+0000","last_change":"2024-04-02T16:11:32.172101+0000","last_active":"2024-04-02T16:11:33.159396+0000","last_peered":"2024-04-02T16:11:33.159396+0000","last_clean":"2024-04-02T16:11:33.159396+0000","last_became_active":"2024-04-02T16:11:32.169192+0000","last_became_peered":"2024-04-02T16:11:32.169192+0000","last_unstale":"2024-04-02T16:11:33.159396+0000","last_undegraded":"2024-04-02T16:11:33.159396+0000","last_fullsized":"2024-04-02T16:11:33.159396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892119+0000","last_change":"2024-04-02T16:11:31.118803+0000","last_active":"2024-04-02T16:11:34.892119+0000","last_peered":"2024-04-02T16:11:34.892119+0000","last_clean":"2024-04-02T16:11:34.892119+0000","last_became_active":"2024-04-02T16:11:31.118400+0000","last_became_peered":"2024-04-02T16:11:31.118400+0000","last_unstale":"2024-04-02T16:11:34.892119+0000","last_undegraded":"2024-04-02T16:11:34.892119+0000","last_fullsized":"2024-04-02T16:11:34.892119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497571+0000","last_change":"2024-04-02T16:11:32.175909+0000","last_active":"2024-04-02T16:11:33.497571+0000","last_peered":"2024-04-02T16:11:33.497571+0000","last_clean":"2024-04-02T16:11:33.497571+0000","last_became_active":"2024-04-02T16:11:32.175629+0000","last_became_peered":"2024-04-02T16:11:32.175629+0000","last_unstale":"2024-04-02T16:11:33.497571+0000","last_undegraded":"2024-04-02T16:11:33.497571+0000","last_fullsized":"2024-04-02T16:11:33.497571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496798+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:11:33.496798+0000","last_peered":"2024-04-02T16:11:33.496798+0000","last_clean":"2024-04-02T16:11:33.496798+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:11:33.496798+0000","last_undegraded":"2024-04-02T16:11:33.496798+0000","last_fullsized":"2024-04-02T16:11:33.496798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158329+0000","last_change":"2024-04-02T16:11:31.114855+0000","last_active":"2024-04-02T16:11:33.158329+0000","last_peered":"2024-04-02T16:11:33.158329+0000","last_clean":"2024-04-02T16:11:33.158329+0000","last_became_active":"2024-04-02T16:11:31.114136+0000","last_became_peered":"2024-04-02T16:11:31.114136+0000","last_unstale":"2024-04-02T16:11:33.158329+0000","last_undegraded":"2024-04-02T16:11:33.158329+0000","last_fullsized":"2024-04-02T16:11:33.158329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891625+0000","last_change":"2024-04-02T16:11:32.177918+0000","last_active":"2024-04-02T16:11:34.891625+0000","last_peered":"2024-04-02T16:11:34.891625+0000","last_clean":"2024-04-02T16:11:34.891625+0000","last_became_active":"2024-04-02T16:11:32.177523+0000","last_became_peered":"2024-04-02T16:11:32.177523+0000","last_unstale":"2024-04-02T16:11:34.891625+0000","last_undegraded":"2024-04-02T16:11:34.891625+0000","last_fullsized":"2024-04-02T16:11:34.891625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496680+0000","last_change":"2024-04-02T16:11:31.147127+0000","last_active":"2024-04-02T16:11:33.496680+0000","last_peered":"2024-04-02T16:11:33.496680+0000","last_clean":"2024-04-02T16:11:33.496680+0000","last_became_active":"2024-04-02T16:11:31.146323+0000","last_became_peered":"2024-04-02T16:11:31.146323+0000","last_unstale":"2024-04-02T16:11:33.496680+0000","last_undegraded":"2024-04-02T16:11:33.496680+0000","last_fullsized":"2024-04-02T16:11:33.496680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890757+0000","last_change":"2024-04-02T16:11:32.152084+0000","last_active":"2024-04-02T16:11:34.890757+0000","last_peered":"2024-04-02T16:11:34.890757+0000","last_clean":"2024-04-02T16:11:34.890757+0000","last_became_active":"2024-04-02T16:11:32.151565+0000","last_became_peered":"2024-04-02T16:11:32.151565+0000","last_unstale":"2024-04-02T16:11:34.890757+0000","last_undegraded":"2024-04-02T16:11:34.890757+0000","last_fullsized":"2024-04-02T16:11:34.890757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:28:10.993274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890041+0000","last_change":"2024-04-02T16:11:31.112830+0000","last_active":"2024-04-02T16:11:34.890041+0000","last_peered":"2024-04-02T16:11:34.890041+0000","last_clean":"2024-04-02T16:11:34.890041+0000","last_became_active":"2024-04-02T16:11:31.112472+0000","last_became_peered":"2024-04-02T16:11:31.112472+0000","last_unstale":"2024-04-02T16:11:34.890041+0000","last_undegraded":"2024-04-02T16:11:34.890041+0000","last_fullsized":"2024-04-02T16:11:34.890041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156504+0000","last_change":"2024-04-02T16:11:32.152287+0000","last_active":"2024-04-02T16:11:33.156504+0000","last_peered":"2024-04-02T16:11:33.156504+0000","last_clean":"2024-04-02T16:11:33.156504+0000","last_became_active":"2024-04-02T16:11:32.151559+0000","last_became_peered":"2024-04-02T16:11:32.151559+0000","last_unstale":"2024-04-02T16:11:33.156504+0000","last_undegraded":"2024-04-02T16:11:33.156504+0000","last_fullsized":"2024-04-02T16:11:33.156504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158997+0000","last_change":"2024-04-02T16:11:31.131581+0000","last_active":"2024-04-02T16:11:33.158997+0000","last_peered":"2024-04-02T16:11:33.158997+0000","last_clean":"2024-04-02T16:11:33.158997+0000","last_became_active":"2024-04-02T16:11:31.130730+0000","last_became_peered":"2024-04-02T16:11:31.130730+0000","last_unstale":"2024-04-02T16:11:33.158997+0000","last_undegraded":"2024-04-02T16:11:33.158997+0000","last_fullsized":"2024-04-02T16:11:33.158997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157459+0000","last_change":"2024-04-02T16:11:32.159434+0000","last_active":"2024-04-02T16:11:33.157459+0000","last_peered":"2024-04-02T16:11:33.157459+0000","last_clean":"2024-04-02T16:11:33.157459+0000","last_became_active":"2024-04-02T16:11:32.159056+0000","last_became_peered":"2024-04-02T16:11:32.159056+0000","last_unstale":"2024-04-02T16:11:33.157459+0000","last_undegraded":"2024-04-02T16:11:33.157459+0000","last_fullsized":"2024-04-02T16:11:33.157459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156640+0000","last_change":"2024-04-02T16:11:31.131364+0000","last_active":"2024-04-02T16:11:33.156640+0000","last_peered":"2024-04-02T16:11:33.156640+0000","last_clean":"2024-04-02T16:11:33.156640+0000","last_became_active":"2024-04-02T16:11:31.130814+0000","last_became_peered":"2024-04-02T16:11:31.130814+0000","last_unstale":"2024-04-02T16:11:33.156640+0000","last_undegraded":"2024-04-02T16:11:33.156640+0000","last_fullsized":"2024-04-02T16:11:33.156640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160072+0000","last_change":"2024-04-02T16:11:32.168617+0000","last_active":"2024-04-02T16:11:33.160072+0000","last_peered":"2024-04-02T16:11:33.160072+0000","last_clean":"2024-04-02T16:11:33.160072+0000","last_became_active":"2024-04-02T16:11:32.168318+0000","last_became_peered":"2024-04-02T16:11:32.168318+0000","last_unstale":"2024-04-02T16:11:33.160072+0000","last_undegraded":"2024-04-02T16:11:33.160072+0000","last_fullsized":"2024-04-02T16:11:33.160072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497910+0000","last_change":"2024-04-02T16:11:31.150504+0000","last_active":"2024-04-02T16:11:33.497910+0000","last_peered":"2024-04-02T16:11:33.497910+0000","last_clean":"2024-04-02T16:11:33.497910+0000","last_became_active":"2024-04-02T16:11:31.149962+0000","last_became_peered":"2024-04-02T16:11:31.149962+0000","last_unstale":"2024-04-02T16:11:33.497910+0000","last_undegraded":"2024-04-02T16:11:33.497910+0000","last_fullsized":"2024-04-02T16:11:33.497910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:35.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156819+0000","last_change":"2024-04-02T16:11:32.157220+0000","last_active":"2024-04-02T16:11:33.156819+0000","last_peered":"2024-04-02T16:11:33.156819+0000","last_clean":"2024-04-02T16:11:33.156819+0000","last_became_active":"2024-04-02T16:11:32.156725+0000","last_became_peered":"2024-04-02T16:11:32.156725+0000","last_unstale":"2024-04-02T16:11:33.156819+0000","last_undegraded":"2024-04-02T16:11:33.156819+0000","last_fullsized":"2024-04-02T16:11:33.156819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.159254+0000","last_change":"2024-04-02T16:11:31.130016+0000","last_active":"2024-04-02T16:11:34.159254+0000","last_peered":"2024-04-02T16:11:34.159254+0000","last_clean":"2024-04-02T16:11:34.159254+0000","last_became_active":"2024-04-02T16:11:31.129578+0000","last_became_peered":"2024-04-02T16:11:31.129578+0000","last_unstale":"2024-04-02T16:11:34.159254+0000","last_undegraded":"2024-04-02T16:11:34.159254+0000","last_fullsized":"2024-04-02T16:11:34.159254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891591+0000","last_change":"2024-04-02T16:11:32.162159+0000","last_active":"2024-04-02T16:11:34.891591+0000","last_peered":"2024-04-02T16:11:34.891591+0000","last_clean":"2024-04-02T16:11:34.891591+0000","last_became_active":"2024-04-02T16:11:32.161875+0000","last_became_peered":"2024-04-02T16:11:32.161875+0000","last_unstale":"2024-04-02T16:11:34.891591+0000","last_undegraded":"2024-04-02T16:11:34.891591+0000","last_fullsized":"2024-04-02T16:11:34.891591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161008+0000","last_change":"2024-04-02T16:11:31.143750+0000","last_active":"2024-04-02T16:11:34.161008+0000","last_peered":"2024-04-02T16:11:34.161008+0000","last_clean":"2024-04-02T16:11:34.161008+0000","last_became_active":"2024-04-02T16:11:31.142991+0000","last_became_peered":"2024-04-02T16:11:31.142991+0000","last_unstale":"2024-04-02T16:11:34.161008+0000","last_undegraded":"2024-04-02T16:11:34.161008+0000","last_fullsized":"2024-04-02T16:11:34.161008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891190+0000","last_change":"2024-04-02T16:11:32.153539+0000","last_active":"2024-04-02T16:11:34.891190+0000","last_peered":"2024-04-02T16:11:34.891190+0000","last_clean":"2024-04-02T16:11:34.891190+0000","last_became_active":"2024-04-02T16:11:32.152769+0000","last_became_peered":"2024-04-02T16:11:32.152769+0000","last_unstale":"2024-04-02T16:11:34.891190+0000","last_undegraded":"2024-04-02T16:11:34.891190+0000","last_fullsized":"2024-04-02T16:11:34.891190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:27.528611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496646+0000","last_change":"2024-04-02T16:11:31.144365+0000","last_active":"2024-04-02T16:11:33.496646+0000","last_peered":"2024-04-02T16:11:33.496646+0000","last_clean":"2024-04-02T16:11:33.496646+0000","last_became_active":"2024-04-02T16:11:31.143659+0000","last_became_peered":"2024-04-02T16:11:31.143659+0000","last_unstale":"2024-04-02T16:11:33.496646+0000","last_undegraded":"2024-04-02T16:11:33.496646+0000","last_fullsized":"2024-04-02T16:11:33.496646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157111+0000","last_change":"2024-04-02T16:11:32.153561+0000","last_active":"2024-04-02T16:11:33.157111+0000","last_peered":"2024-04-02T16:11:33.157111+0000","last_clean":"2024-04-02T16:11:33.157111+0000","last_became_active":"2024-04-02T16:11:32.148826+0000","last_became_peered":"2024-04-02T16:11:32.148826+0000","last_unstale":"2024-04-02T16:11:33.157111+0000","last_undegraded":"2024-04-02T16:11:33.157111+0000","last_fullsized":"2024-04-02T16:11:33.157111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156589+0000","last_change":"2024-04-02T16:11:31.144769+0000","last_active":"2024-04-02T16:11:33.156589+0000","last_peered":"2024-04-02T16:11:33.156589+0000","last_clean":"2024-04-02T16:11:33.156589+0000","last_became_active":"2024-04-02T16:11:31.144386+0000","last_became_peered":"2024-04-02T16:11:31.144386+0000","last_unstale":"2024-04-02T16:11:33.156589+0000","last_undegraded":"2024-04-02T16:11:33.156589+0000","last_fullsized":"2024-04-02T16:11:33.156589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890632+0000","last_change":"2024-04-02T16:11:32.160650+0000","last_active":"2024-04-02T16:11:34.890632+0000","last_peered":"2024-04-02T16:11:34.890632+0000","last_clean":"2024-04-02T16:11:34.890632+0000","last_became_active":"2024-04-02T16:11:32.160239+0000","last_became_peered":"2024-04-02T16:11:32.160239+0000","last_unstale":"2024-04-02T16:11:34.890632+0000","last_undegraded":"2024-04-02T16:11:34.890632+0000","last_fullsized":"2024-04-02T16:11:34.890632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157210+0000","last_change":"2024-04-02T16:11:31.119214+0000","last_active":"2024-04-02T16:11:33.157210+0000","last_peered":"2024-04-02T16:11:33.157210+0000","last_clean":"2024-04-02T16:11:33.157210+0000","last_became_active":"2024-04-02T16:11:31.118885+0000","last_became_peered":"2024-04-02T16:11:31.118885+0000","last_unstale":"2024-04-02T16:11:33.157210+0000","last_undegraded":"2024-04-02T16:11:33.157210+0000","last_fullsized":"2024-04-02T16:11:33.157210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890913+0000","last_change":"2024-04-02T16:11:32.165991+0000","last_active":"2024-04-02T16:11:34.890913+0000","last_peered":"2024-04-02T16:11:34.890913+0000","last_clean":"2024-04-02T16:11:34.890913+0000","last_became_active":"2024-04-02T16:11:32.165617+0000","last_became_peered":"2024-04-02T16:11:32.165617+0000","last_unstale":"2024-04-02T16:11:34.890913+0000","last_undegraded":"2024-04-02T16:11:34.890913+0000","last_fullsized":"2024-04-02T16:11:34.890913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497809+0000","last_change":"2024-04-02T16:11:31.150660+0000","last_active":"2024-04-02T16:11:33.497809+0000","last_peered":"2024-04-02T16:11:33.497809+0000","last_clean":"2024-04-02T16:11:33.497809+0000","last_became_active":"2024-04-02T16:11:31.149922+0000","last_became_peered":"2024-04-02T16:11:31.149922+0000","last_unstale":"2024-04-02T16:11:33.497809+0000","last_undegraded":"2024-04-02T16:11:33.497809+0000","last_fullsized":"2024-04-02T16:11:33.497809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:54.118266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160589+0000","last_change":"2024-04-02T16:11:32.166809+0000","last_active":"2024-04-02T16:11:34.160589+0000","last_peered":"2024-04-02T16:11:34.160589+0000","last_clean":"2024-04-02T16:11:34.160589+0000","last_became_active":"2024-04-02T16:11:32.164926+0000","last_became_peered":"2024-04-02T16:11:32.164926+0000","last_unstale":"2024-04-02T16:11:34.160589+0000","last_undegraded":"2024-04-02T16:11:34.160589+0000","last_fullsized":"2024-04-02T16:11:34.160589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159873+0000","last_change":"2024-04-02T16:11:32.146387+0000","last_active":"2024-04-02T16:11:33.159873+0000","last_peered":"2024-04-02T16:11:33.159873+0000","last_clean":"2024-04-02T16:11:33.159873+0000","last_became_active":"2024-04-02T16:11:32.145655+0000","last_became_peered":"2024-04-02T16:11:32.145655+0000","last_unstale":"2024-04-02T16:11:33.159873+0000","last_undegraded":"2024-04-02T16:11:33.159873+0000","last_fullsized":"2024-04-02T16:11:33.159873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497313+0000","last_change":"2024-04-02T16:11:31.147426+0000","last_active":"2024-04-02T16:11:33.497313+0000","last_peered":"2024-04-02T16:11:33.497313+0000","last_clean":"2024-04-02T16:11:33.497313+0000","last_became_active":"2024-04-02T16:11:31.146661+0000","last_became_peered":"2024-04-02T16:11:31.146661+0000","last_unstale":"2024-04-02T16:11:33.497313+0000","last_undegraded":"2024-04-02T16:11:33.497313+0000","last_fullsized":"2024-04-02T16:11:33.497313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159146+0000","last_change":"2024-04-02T16:11:32.163890+0000","last_active":"2024-04-02T16:11:33.159146+0000","last_peered":"2024-04-02T16:11:33.159146+0000","last_clean":"2024-04-02T16:11:33.159146+0000","last_became_active":"2024-04-02T16:11:32.163585+0000","last_became_peered":"2024-04-02T16:11:32.163585+0000","last_unstale":"2024-04-02T16:11:33.159146+0000","last_undegraded":"2024-04-02T16:11:33.159146+0000","last_fullsized":"2024-04-02T16:11:33.159146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497629+0000","last_change":"2024-04-02T16:11:31.151627+0000","last_active":"2024-04-02T16:11:33.497629+0000","last_peered":"2024-04-02T16:11:33.497629+0000","last_clean":"2024-04-02T16:11:33.497629+0000","last_became_active":"2024-04-02T16:11:31.151077+0000","last_became_peered":"2024-04-02T16:11:31.151077+0000","last_unstale":"2024-04-02T16:11:33.497629+0000","last_undegraded":"2024-04-02T16:11:33.497629+0000","last_fullsized":"2024-04-02T16:11:33.497629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:47.686560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893170+0000","last_change":"2024-04-02T16:11:32.153198+0000","last_active":"2024-04-02T16:11:34.893170+0000","last_peered":"2024-04-02T16:11:34.893170+0000","last_clean":"2024-04-02T16:11:34.893170+0000","last_became_active":"2024-04-02T16:11:32.149403+0000","last_became_peered":"2024-04-02T16:11:32.149403+0000","last_unstale":"2024-04-02T16:11:34.893170+0000","last_undegraded":"2024-04-02T16:11:34.893170+0000","last_fullsized":"2024-04-02T16:11:34.893170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892000+0000","last_change":"2024-04-02T16:11:31.128936+0000","last_active":"2024-04-02T16:11:34.892000+0000","last_peered":"2024-04-02T16:11:34.892000+0000","last_clean":"2024-04-02T16:11:34.892000+0000","last_became_active":"2024-04-02T16:11:31.128574+0000","last_became_peered":"2024-04-02T16:11:31.128574+0000","last_unstale":"2024-04-02T16:11:34.892000+0000","last_undegraded":"2024-04-02T16:11:34.892000+0000","last_fullsized":"2024-04-02T16:11:34.892000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890787+0000","last_change":"2024-04-02T16:11:32.170441+0000","last_active":"2024-04-02T16:11:34.890787+0000","last_peered":"2024-04-02T16:11:34.890787+0000","last_clean":"2024-04-02T16:11:34.890787+0000","last_became_active":"2024-04-02T16:11:32.167070+0000","last_became_peered":"2024-04-02T16:11:32.167070+0000","last_unstale":"2024-04-02T16:11:34.890787+0000","last_undegraded":"2024-04-02T16:11:34.890787+0000","last_fullsized":"2024-04-02T16:11:34.890787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893653+0000","last_change":"2024-04-02T16:11:31.142814+0000","last_active":"2024-04-02T16:11:34.893653+0000","last_peered":"2024-04-02T16:11:34.893653+0000","last_clean":"2024-04-02T16:11:34.893653+0000","last_became_active":"2024-04-02T16:11:31.142568+0000","last_became_peered":"2024-04-02T16:11:31.142568+0000","last_unstale":"2024-04-02T16:11:34.893653+0000","last_undegraded":"2024-04-02T16:11:34.893653+0000","last_fullsized":"2024-04-02T16:11:34.893653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:11:41.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:11:41.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.856+0000 7eff6103d640 1 Processor -- start 2024-04-02T16:11:41.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff6103d640 1 -- start start 2024-04-02T16:11:41.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff6103d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7eff5c02c460 0x7eff5c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff6103d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c06d5e0 con 0x7eff5c06e510 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff6103d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c06d8c0 con 0x7eff5c0b8260 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff6103d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c06dc30 con 0x7eff5c02c460 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff5c06e510 0x7eff5c0b7b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50952/0 (socket says 172.21.15.40:50952) 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5a59c640 1 -- 172.21.15.40:0/2556633913 learned_addr learned my addr 172.21.15.40:0/2556633913 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097433294 0 0) 0x7eff5c06d5e0 con 0x7eff5c06e510 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/2556633913 >> v2:172.21.15.204:3300/0 conn(0x7eff5c02c460 0x7eff5c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff44003530 con 0x7eff5c06e510 2024-04-02T16:11:41.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1084496782 0 0) 0x7eff44003530 con 0x7eff5c06e510 2024-04-02T16:11:41.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 >> v2:172.21.15.204:3300/0 conn(0x7eff5c02c460 msgr2=0x7eff5c06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 --2- 172.21.15.40:0/2556633913 >> v2:172.21.15.204:3300/0 conn(0x7eff5c02c460 0x7eff5c06cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 >> v1:172.21.15.204:6789/0 conn(0x7eff5c0b8260 legacy=0x7eff5c147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c06d300 con 0x7eff5c06e510 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7eff50001680 con 0x7eff5c06e510 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 >> v1:172.21.15.40:6789/0 conn(0x7eff5c06e510 legacy=0x7eff5c0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 0x7eff44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff44004810 0x7eff44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440052f0 0x7eff4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400c190 con 0x7eff440039c0 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff4400c340 con 0x7eff440052f0 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff4400c540 con 0x7eff44004810 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 0x7eff44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5a59c640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff44004810 0x7eff44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5b59e640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440052f0 0x7eff4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 -- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff44004810 msgr2=0x7eff44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff44004810 0x7eff44004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 -- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440052f0 msgr2=0x7eff4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440052f0 0x7eff4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5ad9d640 1 -- 172.21.15.40:0/2556633913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff44003530 con 0x7eff440039c0 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5b59e640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440052f0 0x7eff4400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.860+0000 7eff5b59e640 1 -- 172.21.15.40:0/2556633913 reap_dead start 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 0x7eff44003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009a90 tx=0x7eff4c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93059499daaa24de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0b2070 con 0x7eff440039c0 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c0a5510 con 0x7eff440039c0 2024-04-02T16:11:41.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff59d9b640 1 -- 172.21.15.40:0/2556633913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c009690 con 0x7eff440039c0 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 msgr2=0x7eff44003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 0x7eff44003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009a90 tx=0x7eff4c01c9f0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/2556633913 shutdown_connections 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 --2- 172.21.15.40:0/2556633913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff440039c0 0x7eff44003de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/2556633913 >> 172.21.15.40:0/2556633913 conn(0x7eff5c0b5fb0 msgr2=0x7eff5c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/2556633913 shutdown_connections 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/2556633913 wait complete. 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 Processor -- start 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- start start 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440039c0 0x7eff5c072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 0x7eff5c09c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c14ba20 con 0x7eff5c1e3c90 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c14b8a0 con 0x7eff440039c0 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c14bba0 con 0x7eff5c0b3050 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37536/0 (socket says 172.21.15.40:37536) 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 -- 172.21.15.40:0/644884150 learned_addr learned my addr 172.21.15.40:0/644884150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440039c0 0x7eff5c072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5a59c640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 0x7eff5c09c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 -- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 msgr2=0x7eff5c09c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 0x7eff5c09c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 -- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440039c0 msgr2=0x7eff5c072f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440039c0 0x7eff5c072f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 -- 172.21.15.40:0/644884150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c09da40 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5a59c640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 0x7eff5c09c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff5b59e640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0b3a90 tx=0x7eff5400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54018070 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff54003d90 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c1e4110 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54005330 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.864+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c0b2850 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c0b2b80 con 0x7eff5c1e3c90 2024-04-02T16:11:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff5401b070 con 0x7eff5c1e3c90 2024-04-02T16:11:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff437fe640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 0x7eff4806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff54085f30 con 0x7eff5c1e3c90 2024-04-02T16:11:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 0x7eff4806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:41.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.868+0000 7eff5ad9d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 0x7eff4806a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0af000 tx=0x7eff4c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:41.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:41.872+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7eff54056580 con 0x7eff5c1e3c90 2024-04-02T16:11:42.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.032+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff5c068d00 con 0x7eff480683b0 2024-04-02T16:11:42.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.048+0000 7eff437fe640 1 -- 172.21.15.40:0/644884150 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7eff5c068d00 con 0x7eff480683b0 2024-04-02T16:11:42.057 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:42.058 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:11:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 msgr2=0x7eff4806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 0x7eff4806a870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0af000 tx=0x7eff4c0046d0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 msgr2=0x7eff5c0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0b3a90 tx=0x7eff5400d9f0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 shutdown_connections 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff5c0b3050 0x7eff5c09c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff480683b0 0x7eff4806a870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff5c1e3c90 0x7eff5c0b08b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 --2- 172.21.15.40:0/644884150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff440039c0 0x7eff5c072f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 >> 172.21.15.40:0/644884150 conn(0x7eff5c0b5fb0 msgr2=0x7eff5c0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 shutdown_connections 2024-04-02T16:11:42.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.052+0000 7eff6103d640 1 -- 172.21.15.40:0/644884150 wait complete. 2024-04-02T16:11:42.076 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-02T16:11:41.851285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.725210"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497949+0000","last_change":"2024-04-02T16:11:32.176361+0000","last_active":"2024-04-02T16:11:33.497949+0000","last_peered":"2024-04-02T16:11:33.497949+0000","last_clean":"2024-04-02T16:11:33.497949+0000","last_became_active":"2024-04-02T16:11:32.176117+0000","last_became_peered":"2024-04-02T16:11:32.176117+0000","last_unstale":"2024-04-02T16:11:33.497949+0000","last_undegraded":"2024-04-02T16:11:33.497949+0000","last_fullsized":"2024-04-02T16:11:33.497949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:03.664122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156872+0000","last_change":"2024-04-02T16:11:32.165208+0000","last_active":"2024-04-02T16:11:33.156872+0000","last_peered":"2024-04-02T16:11:33.156872+0000","last_clean":"2024-04-02T16:11:33.156872+0000","last_became_active":"2024-04-02T16:11:32.164869+0000","last_became_peered":"2024-04-02T16:11:32.164869+0000","last_unstale":"2024-04-02T16:11:33.156872+0000","last_undegraded":"2024-04-02T16:11:33.156872+0000","last_fullsized":"2024-04-02T16:11:33.156872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892407+0000","last_change":"2024-04-02T16:11:32.146435+0000","last_active":"2024-04-02T16:11:34.892407+0000","last_peered":"2024-04-02T16:11:34.892407+0000","last_clean":"2024-04-02T16:11:34.892407+0000","last_became_active":"2024-04-02T16:11:32.145704+0000","last_became_peered":"2024-04-02T16:11:32.145704+0000","last_unstale":"2024-04-02T16:11:34.892407+0000","last_undegraded":"2024-04-02T16:11:34.892407+0000","last_fullsized":"2024-04-02T16:11:34.892407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158980+0000","last_change":"2024-04-02T16:11:32.172527+0000","last_active":"2024-04-02T16:11:33.158980+0000","last_peered":"2024-04-02T16:11:33.158980+0000","last_clean":"2024-04-02T16:11:33.158980+0000","last_became_active":"2024-04-02T16:11:32.170021+0000","last_became_peered":"2024-04-02T16:11:32.170021+0000","last_unstale":"2024-04-02T16:11:33.158980+0000","last_undegraded":"2024-04-02T16:11:33.158980+0000","last_fullsized":"2024-04-02T16:11:33.158980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497274+0000","last_change":"2024-04-02T16:11:32.171344+0000","last_active":"2024-04-02T16:11:33.497274+0000","last_peered":"2024-04-02T16:11:33.497274+0000","last_clean":"2024-04-02T16:11:33.497274+0000","last_became_active":"2024-04-02T16:11:32.170838+0000","last_became_peered":"2024-04-02T16:11:32.170838+0000","last_unstale":"2024-04-02T16:11:33.497274+0000","last_undegraded":"2024-04-02T16:11:33.497274+0000","last_fullsized":"2024-04-02T16:11:33.497274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158169+0000","last_change":"2024-04-02T16:11:32.160886+0000","last_active":"2024-04-02T16:11:33.158169+0000","last_peered":"2024-04-02T16:11:33.158169+0000","last_clean":"2024-04-02T16:11:33.158169+0000","last_became_active":"2024-04-02T16:11:32.159998+0000","last_became_peered":"2024-04-02T16:11:32.159998+0000","last_unstale":"2024-04-02T16:11:33.158169+0000","last_undegraded":"2024-04-02T16:11:33.158169+0000","last_fullsized":"2024-04-02T16:11:33.158169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496848+0000","last_change":"2024-04-02T16:11:32.172231+0000","last_active":"2024-04-02T16:11:33.496848+0000","last_peered":"2024-04-02T16:11:33.496848+0000","last_clean":"2024-04-02T16:11:33.496848+0000","last_became_active":"2024-04-02T16:11:32.171852+0000","last_became_peered":"2024-04-02T16:11:32.171852+0000","last_unstale":"2024-04-02T16:11:33.496848+0000","last_undegraded":"2024-04-02T16:11:33.496848+0000","last_fullsized":"2024-04-02T16:11:33.496848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889767+0000","last_change":"2024-04-02T16:11:32.171342+0000","last_active":"2024-04-02T16:11:34.889767+0000","last_peered":"2024-04-02T16:11:34.889767+0000","last_clean":"2024-04-02T16:11:34.889767+0000","last_became_active":"2024-04-02T16:11:32.171027+0000","last_became_peered":"2024-04-02T16:11:32.171027+0000","last_unstale":"2024-04-02T16:11:34.889767+0000","last_undegraded":"2024-04-02T16:11:34.889767+0000","last_fullsized":"2024-04-02T16:11:34.889767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:34:40.739755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498216+0000","last_change":"2024-04-02T16:11:32.171461+0000","last_active":"2024-04-02T16:11:33.498216+0000","last_peered":"2024-04-02T16:11:33.498216+0000","last_clean":"2024-04-02T16:11:33.498216+0000","last_became_active":"2024-04-02T16:11:32.170971+0000","last_became_peered":"2024-04-02T16:11:32.170971+0000","last_unstale":"2024-04-02T16:11:33.498216+0000","last_undegraded":"2024-04-02T16:11:33.498216+0000","last_fullsized":"2024-04-02T16:11:33.498216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:25.000722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161307+0000","last_change":"2024-04-02T16:11:32.164006+0000","last_active":"2024-04-02T16:11:34.161307+0000","last_peered":"2024-04-02T16:11:34.161307+0000","last_clean":"2024-04-02T16:11:34.161307+0000","last_became_active":"2024-04-02T16:11:32.163712+0000","last_became_peered":"2024-04-02T16:11:32.163712+0000","last_unstale":"2024-04-02T16:11:34.161307+0000","last_undegraded":"2024-04-02T16:11:34.161307+0000","last_fullsized":"2024-04-02T16:11:34.161307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160963+0000","last_change":"2024-04-02T16:11:32.162643+0000","last_active":"2024-04-02T16:11:34.160963+0000","last_peered":"2024-04-02T16:11:34.160963+0000","last_clean":"2024-04-02T16:11:34.160963+0000","last_became_active":"2024-04-02T16:11:32.162329+0000","last_became_peered":"2024-04-02T16:11:32.162329+0000","last_unstale":"2024-04-02T16:11:34.160963+0000","last_undegraded":"2024-04-02T16:11:34.160963+0000","last_fullsized":"2024-04-02T16:11:34.160963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158378+0000","last_change":"2024-04-02T16:11:32.150507+0000","last_active":"2024-04-02T16:11:33.158378+0000","last_peered":"2024-04-02T16:11:33.158378+0000","last_clean":"2024-04-02T16:11:33.158378+0000","last_became_active":"2024-04-02T16:11:32.149817+0000","last_became_peered":"2024-04-02T16:11:32.149817+0000","last_unstale":"2024-04-02T16:11:33.158378+0000","last_undegraded":"2024-04-02T16:11:33.158378+0000","last_fullsized":"2024-04-02T16:11:33.158378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890701+0000","last_change":"2024-04-02T16:11:32.171214+0000","last_active":"2024-04-02T16:11:34.890701+0000","last_peered":"2024-04-02T16:11:34.890701+0000","last_clean":"2024-04-02T16:11:34.890701+0000","last_became_active":"2024-04-02T16:11:32.170528+0000","last_became_peered":"2024-04-02T16:11:32.170528+0000","last_unstale":"2024-04-02T16:11:34.890701+0000","last_undegraded":"2024-04-02T16:11:34.890701+0000","last_fullsized":"2024-04-02T16:11:34.890701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891411+0000","last_change":"2024-04-02T16:11:32.148238+0000","last_active":"2024-04-02T16:11:34.891411+0000","last_peered":"2024-04-02T16:11:34.891411+0000","last_clean":"2024-04-02T16:11:34.891411+0000","last_became_active":"2024-04-02T16:11:32.147348+0000","last_became_peered":"2024-04-02T16:11:32.147348+0000","last_unstale":"2024-04-02T16:11:34.891411+0000","last_undegraded":"2024-04-02T16:11:34.891411+0000","last_fullsized":"2024-04-02T16:11:34.891411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:34:40.739755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156485+0000","last_change":"2024-04-02T16:11:32.174912+0000","last_active":"2024-04-02T16:11:33.156485+0000","last_peered":"2024-04-02T16:11:33.156485+0000","last_clean":"2024-04-02T16:11:33.156485+0000","last_became_active":"2024-04-02T16:11:32.174581+0000","last_became_peered":"2024-04-02T16:11:32.174581+0000","last_unstale":"2024-04-02T16:11:33.156485+0000","last_undegraded":"2024-04-02T16:11:33.156485+0000","last_fullsized":"2024-04-02T16:11:33.156485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156962+0000","last_change":"2024-04-02T16:11:32.160602+0000","last_active":"2024-04-02T16:11:33.156962+0000","last_peered":"2024-04-02T16:11:33.156962+0000","last_clean":"2024-04-02T16:11:33.156962+0000","last_became_active":"2024-04-02T16:11:32.160289+0000","last_became_peered":"2024-04-02T16:11:32.160289+0000","last_unstale":"2024-04-02T16:11:33.156962+0000","last_undegraded":"2024-04-02T16:11:33.156962+0000","last_fullsized":"2024-04-02T16:11:33.156962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159486+0000","last_change":"2024-04-02T16:11:32.159580+0000","last_active":"2024-04-02T16:11:33.159486+0000","last_peered":"2024-04-02T16:11:33.159486+0000","last_clean":"2024-04-02T16:11:33.159486+0000","last_became_active":"2024-04-02T16:11:32.159248+0000","last_became_peered":"2024-04-02T16:11:32.159248+0000","last_unstale":"2024-04-02T16:11:33.159486+0000","last_undegraded":"2024-04-02T16:11:33.159486+0000","last_fullsized":"2024-04-02T16:11:33.159486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891860+0000","last_change":"2024-04-02T16:11:32.152173+0000","last_active":"2024-04-02T16:11:34.891860+0000","last_peered":"2024-04-02T16:11:34.891860+0000","last_clean":"2024-04-02T16:11:34.891860+0000","last_became_active":"2024-04-02T16:11:32.150901+0000","last_became_peered":"2024-04-02T16:11:32.150901+0000","last_unstale":"2024-04-02T16:11:34.891860+0000","last_undegraded":"2024-04-02T16:11:34.891860+0000","last_fullsized":"2024-04-02T16:11:34.891860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:29.781008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497849+0000","last_change":"2024-04-02T16:11:32.174590+0000","last_active":"2024-04-02T16:11:33.497849+0000","last_peered":"2024-04-02T16:11:33.497849+0000","last_clean":"2024-04-02T16:11:33.497849+0000","last_became_active":"2024-04-02T16:11:32.174269+0000","last_became_peered":"2024-04-02T16:11:32.174269+0000","last_unstale":"2024-04-02T16:11:33.497849+0000","last_undegraded":"2024-04-02T16:11:33.497849+0000","last_fullsized":"2024-04-02T16:11:33.497849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160975+0000","last_change":"2024-04-02T16:11:32.174103+0000","last_active":"2024-04-02T16:11:34.160975+0000","last_peered":"2024-04-02T16:11:34.160975+0000","last_clean":"2024-04-02T16:11:34.160975+0000","last_became_active":"2024-04-02T16:11:32.173844+0000","last_became_peered":"2024-04-02T16:11:32.173844+0000","last_unstale":"2024-04-02T16:11:34.160975+0000","last_undegraded":"2024-04-02T16:11:34.160975+0000","last_fullsized":"2024-04-02T16:11:34.160975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893151+0000","last_change":"2024-04-02T16:11:32.155248+0000","last_active":"2024-04-02T16:11:34.893151+0000","last_peered":"2024-04-02T16:11:34.893151+0000","last_clean":"2024-04-02T16:11:34.893151+0000","last_became_active":"2024-04-02T16:11:32.154493+0000","last_became_peered":"2024-04-02T16:11:32.154493+0000","last_unstale":"2024-04-02T16:11:34.893151+0000","last_undegraded":"2024-04-02T16:11:34.893151+0000","last_fullsized":"2024-04-02T16:11:34.893151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158188+0000","last_change":"2024-04-02T16:11:32.163843+0000","last_active":"2024-04-02T16:11:33.158188+0000","last_peered":"2024-04-02T16:11:33.158188+0000","last_clean":"2024-04-02T16:11:33.158188+0000","last_became_active":"2024-04-02T16:11:32.163493+0000","last_became_peered":"2024-04-02T16:11:32.163493+0000","last_unstale":"2024-04-02T16:11:33.158188+0000","last_undegraded":"2024-04-02T16:11:33.158188+0000","last_fullsized":"2024-04-02T16:11:33.158188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890013+0000","last_change":"2024-04-02T16:11:32.160644+0000","last_active":"2024-04-02T16:11:34.890013+0000","last_peered":"2024-04-02T16:11:34.890013+0000","last_clean":"2024-04-02T16:11:34.890013+0000","last_became_active":"2024-04-02T16:11:32.160265+0000","last_became_peered":"2024-04-02T16:11:32.160265+0000","last_unstale":"2024-04-02T16:11:34.890013+0000","last_undegraded":"2024-04-02T16:11:34.890013+0000","last_fullsized":"2024-04-02T16:11:34.890013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157349+0000","last_change":"2024-04-02T16:11:32.169639+0000","last_active":"2024-04-02T16:11:33.157349+0000","last_peered":"2024-04-02T16:11:33.157349+0000","last_clean":"2024-04-02T16:11:33.157349+0000","last_became_active":"2024-04-02T16:11:32.166290+0000","last_became_peered":"2024-04-02T16:11:32.166290+0000","last_unstale":"2024-04-02T16:11:33.157349+0000","last_undegraded":"2024-04-02T16:11:33.157349+0000","last_fullsized":"2024-04-02T16:11:33.157349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891192+0000","last_change":"2024-04-02T16:11:32.172667+0000","last_active":"2024-04-02T16:11:34.891192+0000","last_peered":"2024-04-02T16:11:34.891192+0000","last_clean":"2024-04-02T16:11:34.891192+0000","last_became_active":"2024-04-02T16:11:32.171185+0000","last_became_peered":"2024-04-02T16:11:32.171185+0000","last_unstale":"2024-04-02T16:11:34.891192+0000","last_undegraded":"2024-04-02T16:11:34.891192+0000","last_fullsized":"2024-04-02T16:11:34.891192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161390+0000","last_change":"2024-04-02T16:11:32.174173+0000","last_active":"2024-04-02T16:11:34.161390+0000","last_peered":"2024-04-02T16:11:34.161390+0000","last_clean":"2024-04-02T16:11:34.161390+0000","last_became_active":"2024-04-02T16:11:32.173799+0000","last_became_peered":"2024-04-02T16:11:32.173799+0000","last_unstale":"2024-04-02T16:11:34.161390+0000","last_undegraded":"2024-04-02T16:11:34.161390+0000","last_fullsized":"2024-04-02T16:11:34.161390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156692+0000","last_change":"2024-04-02T16:11:32.169323+0000","last_active":"2024-04-02T16:11:33.156692+0000","last_peered":"2024-04-02T16:11:33.156692+0000","last_clean":"2024-04-02T16:11:33.156692+0000","last_became_active":"2024-04-02T16:11:32.169028+0000","last_became_peered":"2024-04-02T16:11:32.169028+0000","last_unstale":"2024-04-02T16:11:33.156692+0000","last_undegraded":"2024-04-02T16:11:33.156692+0000","last_fullsized":"2024-04-02T16:11:33.156692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891206+0000","last_change":"2024-04-02T16:11:32.177772+0000","last_active":"2024-04-02T16:11:34.891206+0000","last_peered":"2024-04-02T16:11:34.891206+0000","last_clean":"2024-04-02T16:11:34.891206+0000","last_became_active":"2024-04-02T16:11:32.177123+0000","last_became_peered":"2024-04-02T16:11:32.177123+0000","last_unstale":"2024-04-02T16:11:34.891206+0000","last_undegraded":"2024-04-02T16:11:34.891206+0000","last_fullsized":"2024-04-02T16:11:34.891206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:15.224251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893485+0000","last_change":"2024-04-02T16:11:32.168117+0000","last_active":"2024-04-02T16:11:34.893485+0000","last_peered":"2024-04-02T16:11:34.893485+0000","last_clean":"2024-04-02T16:11:34.893485+0000","last_became_active":"2024-04-02T16:11:32.167687+0000","last_became_peered":"2024-04-02T16:11:32.167687+0000","last_unstale":"2024-04-02T16:11:34.893485+0000","last_undegraded":"2024-04-02T16:11:34.893485+0000","last_fullsized":"2024-04-02T16:11:34.893485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158588+0000","last_change":"2024-04-02T16:11:32.168501+0000","last_active":"2024-04-02T16:11:33.158588+0000","last_peered":"2024-04-02T16:11:33.158588+0000","last_clean":"2024-04-02T16:11:33.158588+0000","last_became_active":"2024-04-02T16:11:32.168164+0000","last_became_peered":"2024-04-02T16:11:32.168164+0000","last_unstale":"2024-04-02T16:11:33.158588+0000","last_undegraded":"2024-04-02T16:11:33.158588+0000","last_fullsized":"2024-04-02T16:11:33.158588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889779+0000","last_change":"2024-04-02T16:11:32.152124+0000","last_active":"2024-04-02T16:11:34.889779+0000","last_peered":"2024-04-02T16:11:34.889779+0000","last_clean":"2024-04-02T16:11:34.889779+0000","last_became_active":"2024-04-02T16:11:32.150859+0000","last_became_peered":"2024-04-02T16:11:32.150859+0000","last_unstale":"2024-04-02T16:11:34.889779+0000","last_undegraded":"2024-04-02T16:11:34.889779+0000","last_fullsized":"2024-04-02T16:11:34.889779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160495+0000","last_change":"2024-04-02T16:11:32.166672+0000","last_active":"2024-04-02T16:11:34.160495+0000","last_peered":"2024-04-02T16:11:34.160495+0000","last_clean":"2024-04-02T16:11:34.160495+0000","last_became_active":"2024-04-02T16:11:32.165841+0000","last_became_peered":"2024-04-02T16:11:32.165841+0000","last_unstale":"2024-04-02T16:11:34.160495+0000","last_undegraded":"2024-04-02T16:11:34.160495+0000","last_fullsized":"2024-04-02T16:11:34.160495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157143+0000","last_change":"2024-04-02T16:11:32.148450+0000","last_active":"2024-04-02T16:11:33.157143+0000","last_peered":"2024-04-02T16:11:33.157143+0000","last_clean":"2024-04-02T16:11:33.157143+0000","last_became_active":"2024-04-02T16:11:32.147722+0000","last_became_peered":"2024-04-02T16:11:32.147722+0000","last_unstale":"2024-04-02T16:11:33.157143+0000","last_undegraded":"2024-04-02T16:11:33.157143+0000","last_fullsized":"2024-04-02T16:11:33.157143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:44.545404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890986+0000","last_change":"2024-04-02T16:11:31.123742+0000","last_active":"2024-04-02T16:11:34.890986+0000","last_peered":"2024-04-02T16:11:34.890986+0000","last_clean":"2024-04-02T16:11:34.890986+0000","last_became_active":"2024-04-02T16:11:31.123404+0000","last_became_peered":"2024-04-02T16:11:31.123404+0000","last_unstale":"2024-04-02T16:11:34.890986+0000","last_undegraded":"2024-04-02T16:11:34.890986+0000","last_fullsized":"2024-04-02T16:11:34.890986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893703+0000","last_change":"2024-04-02T16:11:32.140773+0000","last_active":"2024-04-02T16:11:34.893703+0000","last_peered":"2024-04-02T16:11:34.893703+0000","last_clean":"2024-04-02T16:11:34.893703+0000","last_became_active":"2024-04-02T16:11:32.140008+0000","last_became_peered":"2024-04-02T16:11:32.140008+0000","last_unstale":"2024-04-02T16:11:34.893703+0000","last_undegraded":"2024-04-02T16:11:34.893703+0000","last_fullsized":"2024-04-02T16:11:34.893703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:08:36.449091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160916+0000","last_change":"2024-04-02T16:11:31.131833+0000","last_active":"2024-04-02T16:11:34.160916+0000","last_peered":"2024-04-02T16:11:34.160916+0000","last_clean":"2024-04-02T16:11:34.160916+0000","last_became_active":"2024-04-02T16:11:31.130705+0000","last_became_peered":"2024-04-02T16:11:31.130705+0000","last_unstale":"2024-04-02T16:11:34.160916+0000","last_undegraded":"2024-04-02T16:11:34.160916+0000","last_fullsized":"2024-04-02T16:11:34.160916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890052+0000","last_change":"2024-04-02T16:11:31.130860+0000","last_active":"2024-04-02T16:11:34.890052+0000","last_peered":"2024-04-02T16:11:34.890052+0000","last_clean":"2024-04-02T16:11:34.890052+0000","last_became_active":"2024-04-02T16:11:31.130391+0000","last_became_peered":"2024-04-02T16:11:31.130391+0000","last_unstale":"2024-04-02T16:11:34.890052+0000","last_undegraded":"2024-04-02T16:11:34.890052+0000","last_fullsized":"2024-04-02T16:11:34.890052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890211+0000","last_change":"2024-04-02T16:11:32.166977+0000","last_active":"2024-04-02T16:11:34.890211+0000","last_peered":"2024-04-02T16:11:34.890211+0000","last_clean":"2024-04-02T16:11:34.890211+0000","last_became_active":"2024-04-02T16:11:32.166690+0000","last_became_peered":"2024-04-02T16:11:32.166690+0000","last_unstale":"2024-04-02T16:11:34.890211+0000","last_undegraded":"2024-04-02T16:11:34.890211+0000","last_fullsized":"2024-04-02T16:11:34.890211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157628+0000","last_change":"2024-04-02T16:11:31.128418+0000","last_active":"2024-04-02T16:11:33.157628+0000","last_peered":"2024-04-02T16:11:33.157628+0000","last_clean":"2024-04-02T16:11:33.157628+0000","last_became_active":"2024-04-02T16:11:31.126329+0000","last_became_peered":"2024-04-02T16:11:31.126329+0000","last_unstale":"2024-04-02T16:11:33.157628+0000","last_undegraded":"2024-04-02T16:11:33.157628+0000","last_fullsized":"2024-04-02T16:11:33.157628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156477+0000","last_change":"2024-04-02T16:11:32.164197+0000","last_active":"2024-04-02T16:11:33.156477+0000","last_peered":"2024-04-02T16:11:33.156477+0000","last_clean":"2024-04-02T16:11:33.156477+0000","last_became_active":"2024-04-02T16:11:32.163440+0000","last_became_peered":"2024-04-02T16:11:32.163440+0000","last_unstale":"2024-04-02T16:11:33.156477+0000","last_undegraded":"2024-04-02T16:11:33.156477+0000","last_fullsized":"2024-04-02T16:11:33.156477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161113+0000","last_change":"2024-04-02T16:11:31.137185+0000","last_active":"2024-04-02T16:11:34.161113+0000","last_peered":"2024-04-02T16:11:34.161113+0000","last_clean":"2024-04-02T16:11:34.161113+0000","last_became_active":"2024-04-02T16:11:31.136386+0000","last_became_peered":"2024-04-02T16:11:31.136386+0000","last_unstale":"2024-04-02T16:11:34.161113+0000","last_undegraded":"2024-04-02T16:11:34.161113+0000","last_fullsized":"2024-04-02T16:11:34.161113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158908+0000","last_change":"2024-04-02T16:11:32.159269+0000","last_active":"2024-04-02T16:11:33.158908+0000","last_peered":"2024-04-02T16:11:33.158908+0000","last_clean":"2024-04-02T16:11:33.158908+0000","last_became_active":"2024-04-02T16:11:32.158906+0000","last_became_peered":"2024-04-02T16:11:32.158906+0000","last_unstale":"2024-04-02T16:11:33.158908+0000","last_undegraded":"2024-04-02T16:11:33.158908+0000","last_fullsized":"2024-04-02T16:11:33.158908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159157+0000","last_change":"2024-04-02T16:11:31.133206+0000","last_active":"2024-04-02T16:11:33.159157+0000","last_peered":"2024-04-02T16:11:33.159157+0000","last_clean":"2024-04-02T16:11:33.159157+0000","last_became_active":"2024-04-02T16:11:31.132012+0000","last_became_peered":"2024-04-02T16:11:31.132012+0000","last_unstale":"2024-04-02T16:11:33.159157+0000","last_undegraded":"2024-04-02T16:11:33.159157+0000","last_fullsized":"2024-04-02T16:11:33.159157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893414+0000","last_change":"2024-04-02T16:11:32.153558+0000","last_active":"2024-04-02T16:11:34.893414+0000","last_peered":"2024-04-02T16:11:34.893414+0000","last_clean":"2024-04-02T16:11:34.893414+0000","last_became_active":"2024-04-02T16:11:32.152697+0000","last_became_peered":"2024-04-02T16:11:32.152697+0000","last_unstale":"2024-04-02T16:11:34.893414+0000","last_undegraded":"2024-04-02T16:11:34.893414+0000","last_fullsized":"2024-04-02T16:11:34.893414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156779+0000","last_change":"2024-04-02T16:11:31.134656+0000","last_active":"2024-04-02T16:11:33.156779+0000","last_peered":"2024-04-02T16:11:33.156779+0000","last_clean":"2024-04-02T16:11:33.156779+0000","last_became_active":"2024-04-02T16:11:31.134191+0000","last_became_peered":"2024-04-02T16:11:31.134191+0000","last_unstale":"2024-04-02T16:11:33.156779+0000","last_undegraded":"2024-04-02T16:11:33.156779+0000","last_fullsized":"2024-04-02T16:11:33.156779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158486+0000","last_change":"2024-04-02T16:11:32.154312+0000","last_active":"2024-04-02T16:11:33.158486+0000","last_peered":"2024-04-02T16:11:33.158486+0000","last_clean":"2024-04-02T16:11:33.158486+0000","last_became_active":"2024-04-02T16:11:32.153984+0000","last_became_peered":"2024-04-02T16:11:32.153984+0000","last_unstale":"2024-04-02T16:11:33.158486+0000","last_undegraded":"2024-04-02T16:11:33.158486+0000","last_fullsized":"2024-04-02T16:11:33.158486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:48.696003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161022+0000","last_change":"2024-04-02T16:11:31.128529+0000","last_active":"2024-04-02T16:11:34.161022+0000","last_peered":"2024-04-02T16:11:34.161022+0000","last_clean":"2024-04-02T16:11:34.161022+0000","last_became_active":"2024-04-02T16:11:31.127760+0000","last_became_peered":"2024-04-02T16:11:31.127760+0000","last_unstale":"2024-04-02T16:11:34.161022+0000","last_undegraded":"2024-04-02T16:11:34.161022+0000","last_fullsized":"2024-04-02T16:11:34.161022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497089+0000","last_change":"2024-04-02T16:11:32.169840+0000","last_active":"2024-04-02T16:11:33.497089+0000","last_peered":"2024-04-02T16:11:33.497089+0000","last_clean":"2024-04-02T16:11:33.497089+0000","last_became_active":"2024-04-02T16:11:32.167352+0000","last_became_peered":"2024-04-02T16:11:32.167352+0000","last_unstale":"2024-04-02T16:11:33.497089+0000","last_undegraded":"2024-04-02T16:11:33.497089+0000","last_fullsized":"2024-04-02T16:11:33.497089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:17.122435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893832+0000","last_change":"2024-04-02T16:11:31.135077+0000","last_active":"2024-04-02T16:11:34.893832+0000","last_peered":"2024-04-02T16:11:34.893832+0000","last_clean":"2024-04-02T16:11:34.893832+0000","last_became_active":"2024-04-02T16:11:31.134519+0000","last_became_peered":"2024-04-02T16:11:31.134519+0000","last_unstale":"2024-04-02T16:11:34.893832+0000","last_undegraded":"2024-04-02T16:11:34.893832+0000","last_fullsized":"2024-04-02T16:11:34.893832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:31:41.376089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891799+0000","last_change":"2024-04-02T16:11:32.155398+0000","last_active":"2024-04-02T16:11:34.891799+0000","last_peered":"2024-04-02T16:11:34.891799+0000","last_clean":"2024-04-02T16:11:34.891799+0000","last_became_active":"2024-04-02T16:11:32.154518+0000","last_became_peered":"2024-04-02T16:11:32.154518+0000","last_unstale":"2024-04-02T16:11:34.891799+0000","last_undegraded":"2024-04-02T16:11:34.891799+0000","last_fullsized":"2024-04-02T16:11:34.891799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:43:20.705547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160202+0000","last_change":"2024-04-02T16:11:31.125926+0000","last_active":"2024-04-02T16:11:33.160202+0000","last_peered":"2024-04-02T16:11:33.160202+0000","last_clean":"2024-04-02T16:11:33.160202+0000","last_became_active":"2024-04-02T16:11:31.124109+0000","last_became_peered":"2024-04-02T16:11:31.124109+0000","last_unstale":"2024-04-02T16:11:33.160202+0000","last_undegraded":"2024-04-02T16:11:33.160202+0000","last_fullsized":"2024-04-02T16:11:33.160202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497615+0000","last_change":"2024-04-02T16:11:32.171153+0000","last_active":"2024-04-02T16:11:33.497615+0000","last_peered":"2024-04-02T16:11:33.497615+0000","last_clean":"2024-04-02T16:11:33.497615+0000","last_became_active":"2024-04-02T16:11:32.170399+0000","last_became_peered":"2024-04-02T16:11:32.170399+0000","last_unstale":"2024-04-02T16:11:33.497615+0000","last_undegraded":"2024-04-02T16:11:33.497615+0000","last_fullsized":"2024-04-02T16:11:33.497615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:49:23.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160399+0000","last_change":"2024-04-02T16:11:31.131680+0000","last_active":"2024-04-02T16:11:34.160399+0000","last_peered":"2024-04-02T16:11:34.160399+0000","last_clean":"2024-04-02T16:11:34.160399+0000","last_became_active":"2024-04-02T16:11:31.129432+0000","last_became_peered":"2024-04-02T16:11:31.129432+0000","last_unstale":"2024-04-02T16:11:34.160399+0000","last_undegraded":"2024-04-02T16:11:34.160399+0000","last_fullsized":"2024-04-02T16:11:34.160399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160538+0000","last_change":"2024-04-02T16:11:32.167516+0000","last_active":"2024-04-02T16:11:34.160538+0000","last_peered":"2024-04-02T16:11:34.160538+0000","last_clean":"2024-04-02T16:11:34.160538+0000","last_became_active":"2024-04-02T16:11:32.167201+0000","last_became_peered":"2024-04-02T16:11:32.167201+0000","last_unstale":"2024-04-02T16:11:34.160538+0000","last_undegraded":"2024-04-02T16:11:34.160538+0000","last_fullsized":"2024-04-02T16:11:34.160538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:21.515778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.889403+0000","last_change":"2024-04-02T16:11:31.117868+0000","last_active":"2024-04-02T16:11:34.889403+0000","last_peered":"2024-04-02T16:11:34.889403+0000","last_clean":"2024-04-02T16:11:34.889403+0000","last_became_active":"2024-04-02T16:11:31.117437+0000","last_became_peered":"2024-04-02T16:11:31.117437+0000","last_unstale":"2024-04-02T16:11:34.889403+0000","last_undegraded":"2024-04-02T16:11:34.889403+0000","last_fullsized":"2024-04-02T16:11:34.889403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497088+0000","last_change":"2024-04-02T16:11:32.178110+0000","last_active":"2024-04-02T16:11:33.497088+0000","last_peered":"2024-04-02T16:11:33.497088+0000","last_clean":"2024-04-02T16:11:33.497088+0000","last_became_active":"2024-04-02T16:11:32.177756+0000","last_became_peered":"2024-04-02T16:11:32.177756+0000","last_unstale":"2024-04-02T16:11:33.497088+0000","last_undegraded":"2024-04-02T16:11:33.497088+0000","last_fullsized":"2024-04-02T16:11:33.497088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:20:41.574727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159238+0000","last_change":"2024-04-02T16:11:31.130795+0000","last_active":"2024-04-02T16:11:33.159238+0000","last_peered":"2024-04-02T16:11:33.159238+0000","last_clean":"2024-04-02T16:11:33.159238+0000","last_became_active":"2024-04-02T16:11:31.129907+0000","last_became_peered":"2024-04-02T16:11:31.129907+0000","last_unstale":"2024-04-02T16:11:33.159238+0000","last_undegraded":"2024-04-02T16:11:33.159238+0000","last_fullsized":"2024-04-02T16:11:33.159238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:27:40.603612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497755+0000","last_change":"2024-04-02T16:11:32.173519+0000","last_active":"2024-04-02T16:11:33.497755+0000","last_peered":"2024-04-02T16:11:33.497755+0000","last_clean":"2024-04-02T16:11:33.497755+0000","last_became_active":"2024-04-02T16:11:32.173159+0000","last_became_peered":"2024-04-02T16:11:32.173159+0000","last_unstale":"2024-04-02T16:11:33.497755+0000","last_undegraded":"2024-04-02T16:11:33.497755+0000","last_fullsized":"2024-04-02T16:11:33.497755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892119+0000","last_change":"2024-04-02T16:11:31.113477+0000","last_active":"2024-04-02T16:11:34.892119+0000","last_peered":"2024-04-02T16:11:34.892119+0000","last_clean":"2024-04-02T16:11:34.892119+0000","last_became_active":"2024-04-02T16:11:31.112346+0000","last_became_peered":"2024-04-02T16:11:31.112346+0000","last_unstale":"2024-04-02T16:11:34.892119+0000","last_undegraded":"2024-04-02T16:11:34.892119+0000","last_fullsized":"2024-04-02T16:11:34.892119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.498157+0000","last_change":"2024-04-02T16:11:32.173602+0000","last_active":"2024-04-02T16:11:33.498157+0000","last_peered":"2024-04-02T16:11:33.498157+0000","last_clean":"2024-04-02T16:11:33.498157+0000","last_became_active":"2024-04-02T16:11:32.173294+0000","last_became_peered":"2024-04-02T16:11:32.173294+0000","last_unstale":"2024-04-02T16:11:33.498157+0000","last_undegraded":"2024-04-02T16:11:33.498157+0000","last_fullsized":"2024-04-02T16:11:33.498157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:08:50.854339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156803+0000","last_change":"2024-04-02T16:11:31.134514+0000","last_active":"2024-04-02T16:11:33.156803+0000","last_peered":"2024-04-02T16:11:33.156803+0000","last_clean":"2024-04-02T16:11:33.156803+0000","last_became_active":"2024-04-02T16:11:31.134123+0000","last_became_peered":"2024-04-02T16:11:31.134123+0000","last_unstale":"2024-04-02T16:11:33.156803+0000","last_undegraded":"2024-04-02T16:11:33.156803+0000","last_fullsized":"2024-04-02T16:11:33.156803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159396+0000","last_change":"2024-04-02T16:11:32.172101+0000","last_active":"2024-04-02T16:11:33.159396+0000","last_peered":"2024-04-02T16:11:33.159396+0000","last_clean":"2024-04-02T16:11:33.159396+0000","last_became_active":"2024-04-02T16:11:32.169192+0000","last_became_peered":"2024-04-02T16:11:32.169192+0000","last_unstale":"2024-04-02T16:11:33.159396+0000","last_undegraded":"2024-04-02T16:11:33.159396+0000","last_fullsized":"2024-04-02T16:11:33.159396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892119+0000","last_change":"2024-04-02T16:11:31.118803+0000","last_active":"2024-04-02T16:11:34.892119+0000","last_peered":"2024-04-02T16:11:34.892119+0000","last_clean":"2024-04-02T16:11:34.892119+0000","last_became_active":"2024-04-02T16:11:31.118400+0000","last_became_peered":"2024-04-02T16:11:31.118400+0000","last_unstale":"2024-04-02T16:11:34.892119+0000","last_undegraded":"2024-04-02T16:11:34.892119+0000","last_fullsized":"2024-04-02T16:11:34.892119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497571+0000","last_change":"2024-04-02T16:11:32.175909+0000","last_active":"2024-04-02T16:11:33.497571+0000","last_peered":"2024-04-02T16:11:33.497571+0000","last_clean":"2024-04-02T16:11:33.497571+0000","last_became_active":"2024-04-02T16:11:32.175629+0000","last_became_peered":"2024-04-02T16:11:32.175629+0000","last_unstale":"2024-04-02T16:11:33.497571+0000","last_undegraded":"2024-04-02T16:11:33.497571+0000","last_fullsized":"2024-04-02T16:11:33.497571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:00.662668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496798+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:11:33.496798+0000","last_peered":"2024-04-02T16:11:33.496798+0000","last_clean":"2024-04-02T16:11:33.496798+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:11:33.496798+0000","last_undegraded":"2024-04-02T16:11:33.496798+0000","last_fullsized":"2024-04-02T16:11:33.496798+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158329+0000","last_change":"2024-04-02T16:11:31.114855+0000","last_active":"2024-04-02T16:11:33.158329+0000","last_peered":"2024-04-02T16:11:33.158329+0000","last_clean":"2024-04-02T16:11:33.158329+0000","last_became_active":"2024-04-02T16:11:31.114136+0000","last_became_peered":"2024-04-02T16:11:31.114136+0000","last_unstale":"2024-04-02T16:11:33.158329+0000","last_undegraded":"2024-04-02T16:11:33.158329+0000","last_fullsized":"2024-04-02T16:11:33.158329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891625+0000","last_change":"2024-04-02T16:11:32.177918+0000","last_active":"2024-04-02T16:11:34.891625+0000","last_peered":"2024-04-02T16:11:34.891625+0000","last_clean":"2024-04-02T16:11:34.891625+0000","last_became_active":"2024-04-02T16:11:32.177523+0000","last_became_peered":"2024-04-02T16:11:32.177523+0000","last_unstale":"2024-04-02T16:11:34.891625+0000","last_undegraded":"2024-04-02T16:11:34.891625+0000","last_fullsized":"2024-04-02T16:11:34.891625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:18:51.849572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496680+0000","last_change":"2024-04-02T16:11:31.147127+0000","last_active":"2024-04-02T16:11:33.496680+0000","last_peered":"2024-04-02T16:11:33.496680+0000","last_clean":"2024-04-02T16:11:33.496680+0000","last_became_active":"2024-04-02T16:11:31.146323+0000","last_became_peered":"2024-04-02T16:11:31.146323+0000","last_unstale":"2024-04-02T16:11:33.496680+0000","last_undegraded":"2024-04-02T16:11:33.496680+0000","last_fullsized":"2024-04-02T16:11:33.496680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:38.862152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890757+0000","last_change":"2024-04-02T16:11:32.152084+0000","last_active":"2024-04-02T16:11:34.890757+0000","last_peered":"2024-04-02T16:11:34.890757+0000","last_clean":"2024-04-02T16:11:34.890757+0000","last_became_active":"2024-04-02T16:11:32.151565+0000","last_became_peered":"2024-04-02T16:11:32.151565+0000","last_unstale":"2024-04-02T16:11:34.890757+0000","last_undegraded":"2024-04-02T16:11:34.890757+0000","last_fullsized":"2024-04-02T16:11:34.890757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:28:10.993274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890041+0000","last_change":"2024-04-02T16:11:31.112830+0000","last_active":"2024-04-02T16:11:34.890041+0000","last_peered":"2024-04-02T16:11:34.890041+0000","last_clean":"2024-04-02T16:11:34.890041+0000","last_became_active":"2024-04-02T16:11:31.112472+0000","last_became_peered":"2024-04-02T16:11:31.112472+0000","last_unstale":"2024-04-02T16:11:34.890041+0000","last_undegraded":"2024-04-02T16:11:34.890041+0000","last_fullsized":"2024-04-02T16:11:34.890041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156504+0000","last_change":"2024-04-02T16:11:32.152287+0000","last_active":"2024-04-02T16:11:33.156504+0000","last_peered":"2024-04-02T16:11:33.156504+0000","last_clean":"2024-04-02T16:11:33.156504+0000","last_became_active":"2024-04-02T16:11:32.151559+0000","last_became_peered":"2024-04-02T16:11:32.151559+0000","last_unstale":"2024-04-02T16:11:33.156504+0000","last_undegraded":"2024-04-02T16:11:33.156504+0000","last_fullsized":"2024-04-02T16:11:33.156504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:11:59.783286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.158997+0000","last_change":"2024-04-02T16:11:31.131581+0000","last_active":"2024-04-02T16:11:33.158997+0000","last_peered":"2024-04-02T16:11:33.158997+0000","last_clean":"2024-04-02T16:11:33.158997+0000","last_became_active":"2024-04-02T16:11:31.130730+0000","last_became_peered":"2024-04-02T16:11:31.130730+0000","last_unstale":"2024-04-02T16:11:33.158997+0000","last_undegraded":"2024-04-02T16:11:33.158997+0000","last_fullsized":"2024-04-02T16:11:33.158997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157459+0000","last_change":"2024-04-02T16:11:32.159434+0000","last_active":"2024-04-02T16:11:33.157459+0000","last_peered":"2024-04-02T16:11:33.157459+0000","last_clean":"2024-04-02T16:11:33.157459+0000","last_became_active":"2024-04-02T16:11:32.159056+0000","last_became_peered":"2024-04-02T16:11:32.159056+0000","last_unstale":"2024-04-02T16:11:33.157459+0000","last_undegraded":"2024-04-02T16:11:33.157459+0000","last_fullsized":"2024-04-02T16:11:33.157459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:27:39.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156640+0000","last_change":"2024-04-02T16:11:31.131364+0000","last_active":"2024-04-02T16:11:33.156640+0000","last_peered":"2024-04-02T16:11:33.156640+0000","last_clean":"2024-04-02T16:11:33.156640+0000","last_became_active":"2024-04-02T16:11:31.130814+0000","last_became_peered":"2024-04-02T16:11:31.130814+0000","last_unstale":"2024-04-02T16:11:33.156640+0000","last_undegraded":"2024-04-02T16:11:33.156640+0000","last_fullsized":"2024-04-02T16:11:33.156640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:02:43.124335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.160072+0000","last_change":"2024-04-02T16:11:32.168617+0000","last_active":"2024-04-02T16:11:33.160072+0000","last_peered":"2024-04-02T16:11:33.160072+0000","last_clean":"2024-04-02T16:11:33.160072+0000","last_became_active":"2024-04-02T16:11:32.168318+0000","last_became_peered":"2024-04-02T16:11:32.168318+0000","last_unstale":"2024-04-02T16:11:33.160072+0000","last_undegraded":"2024-04-02T16:11:33.160072+0000","last_fullsized":"2024-04-02T16:11:33.160072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:11.957592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497910+0000","last_change":"2024-04-02T16:11:31.150504+0000","last_active":"2024-04-02T16:11:33.497910+0000","last_peered":"2024-04-02T16:11:33.497910+0000","last_clean":"2024-04-02T16:11:33.497910+0000","last_became_active":"2024-04-02T16:11:31.149962+0000","last_became_peered":"2024-04-02T16:11:31.149962+0000","last_unstale":"2024-04-02T16:11:33.497910+0000","last_undegraded":"2024-04-02T16:11:33.497910+0000","last_fullsized":"2024-04-02T16:11:33.497910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:35.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156819+0000","last_change":"2024-04-02T16:11:32.157220+0000","last_active":"2024-04-02T16:11:33.156819+0000","last_peered":"2024-04-02T16:11:33.156819+0000","last_clean":"2024-04-02T16:11:33.156819+0000","last_became_active":"2024-04-02T16:11:32.156725+0000","last_became_peered":"2024-04-02T16:11:32.156725+0000","last_unstale":"2024-04-02T16:11:33.156819+0000","last_undegraded":"2024-04-02T16:11:33.156819+0000","last_fullsized":"2024-04-02T16:11:33.156819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.159254+0000","last_change":"2024-04-02T16:11:31.130016+0000","last_active":"2024-04-02T16:11:34.159254+0000","last_peered":"2024-04-02T16:11:34.159254+0000","last_clean":"2024-04-02T16:11:34.159254+0000","last_became_active":"2024-04-02T16:11:31.129578+0000","last_became_peered":"2024-04-02T16:11:31.129578+0000","last_unstale":"2024-04-02T16:11:34.159254+0000","last_undegraded":"2024-04-02T16:11:34.159254+0000","last_fullsized":"2024-04-02T16:11:34.159254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891591+0000","last_change":"2024-04-02T16:11:32.162159+0000","last_active":"2024-04-02T16:11:34.891591+0000","last_peered":"2024-04-02T16:11:34.891591+0000","last_clean":"2024-04-02T16:11:34.891591+0000","last_became_active":"2024-04-02T16:11:32.161875+0000","last_became_peered":"2024-04-02T16:11:32.161875+0000","last_unstale":"2024-04-02T16:11:34.891591+0000","last_undegraded":"2024-04-02T16:11:34.891591+0000","last_fullsized":"2024-04-02T16:11:34.891591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:36.606706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.161008+0000","last_change":"2024-04-02T16:11:31.143750+0000","last_active":"2024-04-02T16:11:34.161008+0000","last_peered":"2024-04-02T16:11:34.161008+0000","last_clean":"2024-04-02T16:11:34.161008+0000","last_became_active":"2024-04-02T16:11:31.142991+0000","last_became_peered":"2024-04-02T16:11:31.142991+0000","last_unstale":"2024-04-02T16:11:34.161008+0000","last_undegraded":"2024-04-02T16:11:34.161008+0000","last_fullsized":"2024-04-02T16:11:34.161008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.891190+0000","last_change":"2024-04-02T16:11:32.153539+0000","last_active":"2024-04-02T16:11:34.891190+0000","last_peered":"2024-04-02T16:11:34.891190+0000","last_clean":"2024-04-02T16:11:34.891190+0000","last_became_active":"2024-04-02T16:11:32.152769+0000","last_became_peered":"2024-04-02T16:11:32.152769+0000","last_unstale":"2024-04-02T16:11:34.891190+0000","last_undegraded":"2024-04-02T16:11:34.891190+0000","last_fullsized":"2024-04-02T16:11:34.891190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:27.528611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.496646+0000","last_change":"2024-04-02T16:11:31.144365+0000","last_active":"2024-04-02T16:11:33.496646+0000","last_peered":"2024-04-02T16:11:33.496646+0000","last_clean":"2024-04-02T16:11:33.496646+0000","last_became_active":"2024-04-02T16:11:31.143659+0000","last_became_peered":"2024-04-02T16:11:31.143659+0000","last_unstale":"2024-04-02T16:11:33.496646+0000","last_undegraded":"2024-04-02T16:11:33.496646+0000","last_fullsized":"2024-04-02T16:11:33.496646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:37:43.535960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157111+0000","last_change":"2024-04-02T16:11:32.153561+0000","last_active":"2024-04-02T16:11:33.157111+0000","last_peered":"2024-04-02T16:11:33.157111+0000","last_clean":"2024-04-02T16:11:33.157111+0000","last_became_active":"2024-04-02T16:11:32.148826+0000","last_became_peered":"2024-04-02T16:11:32.148826+0000","last_unstale":"2024-04-02T16:11:33.157111+0000","last_undegraded":"2024-04-02T16:11:33.157111+0000","last_fullsized":"2024-04-02T16:11:33.157111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:18.154145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.156589+0000","last_change":"2024-04-02T16:11:31.144769+0000","last_active":"2024-04-02T16:11:33.156589+0000","last_peered":"2024-04-02T16:11:33.156589+0000","last_clean":"2024-04-02T16:11:33.156589+0000","last_became_active":"2024-04-02T16:11:31.144386+0000","last_became_peered":"2024-04-02T16:11:31.144386+0000","last_unstale":"2024-04-02T16:11:33.156589+0000","last_undegraded":"2024-04-02T16:11:33.156589+0000","last_fullsized":"2024-04-02T16:11:33.156589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:56:22.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890632+0000","last_change":"2024-04-02T16:11:32.160650+0000","last_active":"2024-04-02T16:11:34.890632+0000","last_peered":"2024-04-02T16:11:34.890632+0000","last_clean":"2024-04-02T16:11:34.890632+0000","last_became_active":"2024-04-02T16:11:32.160239+0000","last_became_peered":"2024-04-02T16:11:32.160239+0000","last_unstale":"2024-04-02T16:11:34.890632+0000","last_undegraded":"2024-04-02T16:11:34.890632+0000","last_fullsized":"2024-04-02T16:11:34.890632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.157210+0000","last_change":"2024-04-02T16:11:31.119214+0000","last_active":"2024-04-02T16:11:33.157210+0000","last_peered":"2024-04-02T16:11:33.157210+0000","last_clean":"2024-04-02T16:11:33.157210+0000","last_became_active":"2024-04-02T16:11:31.118885+0000","last_became_peered":"2024-04-02T16:11:31.118885+0000","last_unstale":"2024-04-02T16:11:33.157210+0000","last_undegraded":"2024-04-02T16:11:33.157210+0000","last_fullsized":"2024-04-02T16:11:33.157210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:04:17.144701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890913+0000","last_change":"2024-04-02T16:11:32.165991+0000","last_active":"2024-04-02T16:11:34.890913+0000","last_peered":"2024-04-02T16:11:34.890913+0000","last_clean":"2024-04-02T16:11:34.890913+0000","last_became_active":"2024-04-02T16:11:32.165617+0000","last_became_peered":"2024-04-02T16:11:32.165617+0000","last_unstale":"2024-04-02T16:11:34.890913+0000","last_undegraded":"2024-04-02T16:11:34.890913+0000","last_fullsized":"2024-04-02T16:11:34.890913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:48:29.888093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497809+0000","last_change":"2024-04-02T16:11:31.150660+0000","last_active":"2024-04-02T16:11:33.497809+0000","last_peered":"2024-04-02T16:11:33.497809+0000","last_clean":"2024-04-02T16:11:33.497809+0000","last_became_active":"2024-04-02T16:11:31.149922+0000","last_became_peered":"2024-04-02T16:11:31.149922+0000","last_unstale":"2024-04-02T16:11:33.497809+0000","last_undegraded":"2024-04-02T16:11:33.497809+0000","last_fullsized":"2024-04-02T16:11:33.497809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:07:54.118266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.160589+0000","last_change":"2024-04-02T16:11:32.166809+0000","last_active":"2024-04-02T16:11:34.160589+0000","last_peered":"2024-04-02T16:11:34.160589+0000","last_clean":"2024-04-02T16:11:34.160589+0000","last_became_active":"2024-04-02T16:11:32.164926+0000","last_became_peered":"2024-04-02T16:11:32.164926+0000","last_unstale":"2024-04-02T16:11:34.160589+0000","last_undegraded":"2024-04-02T16:11:34.160589+0000","last_fullsized":"2024-04-02T16:11:34.160589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159873+0000","last_change":"2024-04-02T16:11:32.146387+0000","last_active":"2024-04-02T16:11:33.159873+0000","last_peered":"2024-04-02T16:11:33.159873+0000","last_clean":"2024-04-02T16:11:33.159873+0000","last_became_active":"2024-04-02T16:11:32.145655+0000","last_became_peered":"2024-04-02T16:11:32.145655+0000","last_unstale":"2024-04-02T16:11:33.159873+0000","last_undegraded":"2024-04-02T16:11:33.159873+0000","last_fullsized":"2024-04-02T16:11:33.159873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:08.467668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497313+0000","last_change":"2024-04-02T16:11:31.147426+0000","last_active":"2024-04-02T16:11:33.497313+0000","last_peered":"2024-04-02T16:11:33.497313+0000","last_clean":"2024-04-02T16:11:33.497313+0000","last_became_active":"2024-04-02T16:11:31.146661+0000","last_became_peered":"2024-04-02T16:11:31.146661+0000","last_unstale":"2024-04-02T16:11:33.497313+0000","last_undegraded":"2024-04-02T16:11:33.497313+0000","last_fullsized":"2024-04-02T16:11:33.497313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:28.771564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.159146+0000","last_change":"2024-04-02T16:11:32.163890+0000","last_active":"2024-04-02T16:11:33.159146+0000","last_peered":"2024-04-02T16:11:33.159146+0000","last_clean":"2024-04-02T16:11:33.159146+0000","last_became_active":"2024-04-02T16:11:32.163585+0000","last_became_peered":"2024-04-02T16:11:32.163585+0000","last_unstale":"2024-04-02T16:11:33.159146+0000","last_undegraded":"2024-04-02T16:11:33.159146+0000","last_fullsized":"2024-04-02T16:11:33.159146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:18:57.945221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:33.497629+0000","last_change":"2024-04-02T16:11:31.151627+0000","last_active":"2024-04-02T16:11:33.497629+0000","last_peered":"2024-04-02T16:11:33.497629+0000","last_clean":"2024-04-02T16:11:33.497629+0000","last_became_active":"2024-04-02T16:11:31.151077+0000","last_became_peered":"2024-04-02T16:11:31.151077+0000","last_unstale":"2024-04-02T16:11:33.497629+0000","last_undegraded":"2024-04-02T16:11:33.497629+0000","last_fullsized":"2024-04-02T16:11:33.497629+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:47.686560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893170+0000","last_change":"2024-04-02T16:11:32.153198+0000","last_active":"2024-04-02T16:11:34.893170+0000","last_peered":"2024-04-02T16:11:34.893170+0000","last_clean":"2024-04-02T16:11:34.893170+0000","last_became_active":"2024-04-02T16:11:32.149403+0000","last_became_peered":"2024-04-02T16:11:32.149403+0000","last_unstale":"2024-04-02T16:11:34.893170+0000","last_undegraded":"2024-04-02T16:11:34.893170+0000","last_fullsized":"2024-04-02T16:11:34.893170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:30:23.274849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.892000+0000","last_change":"2024-04-02T16:11:31.128936+0000","last_active":"2024-04-02T16:11:34.892000+0000","last_peered":"2024-04-02T16:11:34.892000+0000","last_clean":"2024-04-02T16:11:34.892000+0000","last_became_active":"2024-04-02T16:11:31.128574+0000","last_became_peered":"2024-04-02T16:11:31.128574+0000","last_unstale":"2024-04-02T16:11:34.892000+0000","last_undegraded":"2024-04-02T16:11:34.892000+0000","last_fullsized":"2024-04-02T16:11:34.892000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:45:12.467876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.890787+0000","last_change":"2024-04-02T16:11:32.170441+0000","last_active":"2024-04-02T16:11:34.890787+0000","last_peered":"2024-04-02T16:11:34.890787+0000","last_clean":"2024-04-02T16:11:34.890787+0000","last_became_active":"2024-04-02T16:11:32.167070+0000","last_became_peered":"2024-04-02T16:11:32.167070+0000","last_unstale":"2024-04-02T16:11:34.890787+0000","last_undegraded":"2024-04-02T16:11:34.890787+0000","last_fullsized":"2024-04-02T16:11:34.890787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:31.090629+0000","last_clean_scrub_stamp":"2024-04-02T16:11:31.090629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:05:38.925698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-02T16:11:34.893653+0000","last_change":"2024-04-02T16:11:31.142814+0000","last_active":"2024-04-02T16:11:34.893653+0000","last_peered":"2024-04-02T16:11:34.893653+0000","last_clean":"2024-04-02T16:11:34.893653+0000","last_became_active":"2024-04-02T16:11:31.142568+0000","last_became_peered":"2024-04-02T16:11:31.142568+0000","last_unstale":"2024-04-02T16:11:34.893653+0000","last_undegraded":"2024-04-02T16:11:34.893653+0000","last_fullsized":"2024-04-02T16:11:34.893653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:30.081185+0000","last_clean_scrub_stamp":"2024-04-02T16:11:30.081185+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:14:10.656209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:11:42.079 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-02T16:11:42.079 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T16:11:42.080 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-02T16:11:42.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-02T16:11:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 Processor -- start 2024-04-02T16:11:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 -- start start 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8980bb7f0 0x7fe8980b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe89814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8980b31f0 con 0x7fe8980b7480 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe8980b5c70 con 0x7fe8980bb7f0 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe8a05af640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8980bbbd0 con 0x7fe89813fb10 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe89db4a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe8980b7480 0x7fe89813f400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50958/0 (socket says 172.21.15.40:50958) 2024-04-02T16:11:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.224+0000 7fe89db4a640 1 -- 172.21.15.40:0/769615643 learned_addr learned my addr 172.21.15.40:0/769615643 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe89814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89e34b640 1 --2- 172.21.15.40:0/769615643 >> v2:172.21.15.204:3300/0 conn(0x7fe8980bb7f0 0x7fe8980b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111625468 0 0) 0x7fe8980b31f0 con 0x7fe8980b7480 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe884003530 con 0x7fe8980b7480 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3529652306 0 0) 0x7fe884003530 con 0x7fe8980b7480 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 msgr2=0x7fe89814ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe89814ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 >> v2:172.21.15.204:3300/0 conn(0x7fe8980bb7f0 msgr2=0x7fe8980b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 --2- 172.21.15.40:0/769615643 >> v2:172.21.15.204:3300/0 conn(0x7fe8980bb7f0 0x7fe8980b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8980b6410 con 0x7fe8980b7480 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fe894001680 con 0x7fe8980b7480 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 >> v1:172.21.15.40:6789/0 conn(0x7fe8980b7480 legacy=0x7fe89813f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8840039c0 0x7fe884003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe884004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe884004d50 0x7fe8840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980b5c70 con 0x7fe884004d50 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980bbbd0 con 0x7fe8840039c0 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe88400b9d0 con 0x7fe89813fb10 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe884004d50 0x7fe8840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89e34b640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8840039c0 0x7fe884003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89db4a640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe884004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 -- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 msgr2=0x7fe884004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe89813fb10 0x7fe884004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 -- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8840039c0 msgr2=0x7fe884003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8840039c0 0x7fe884003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 -- 172.21.15.40:0/769615643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe884003530 con 0x7fe884004d50 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe884004d50 0x7fe8840051d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe888009b10 tx=0x7fe88801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a331fcd240d53f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b2070 con 0x7fe884004d50 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8880a5660 con 0x7fe884004d50 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89d349640 1 -- 172.21.15.40:0/769615643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880096b0 con 0x7fe884004d50 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 -- 172.21.15.40:0/769615643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe884004d50 msgr2=0x7fe8840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 --2- 172.21.15.40:0/769615643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe884004d50 0x7fe8840051d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe888009b10 tx=0x7fe88801ca70 comp rx=0 tx=0).stop 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe89eb4c640 1 -- 172.21.15.40:0/769615643 reap_dead start 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 -- 172.21.15.40:0/769615643 shutdown_connections 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 -- 172.21.15.40:0/769615643 >> 172.21.15.40:0/769615643 conn(0x7fe8980b7f60 msgr2=0x7fe8980bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 -- 172.21.15.40:0/769615643 shutdown_connections 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.228+0000 7fe8a05af640 1 -- 172.21.15.40:0/769615643 wait complete. 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 Processor -- start 2024-04-02T16:11:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- start start 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 0x7fe8981489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe89813ffc0 0x7fe898140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980b68a0 con 0x7fe898141490 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980b6720 con 0x7fe89813ffc0 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8980b6c60 con 0x7fe884004d50 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37554/0 (socket says 172.21.15.40:37554) 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 -- 172.21.15.40:0/2827974313 learned_addr learned my addr 172.21.15.40:0/2827974313 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89e34b640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 0x7fe8981489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89db4a640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe89813ffc0 0x7fe898140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 -- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 msgr2=0x7fe8981489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 0x7fe8981489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 -- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe89813ffc0 msgr2=0x7fe898140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe89813ffc0 0x7fe898140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 -- 172.21.15.40:0/2827974313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe898141e50 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89e34b640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 0x7fe8981489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe89eb4c640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe888009b10 tx=0x7fe884005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880bb070 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe888000e50 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8880b2040 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981e0630 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981e0340 con 0x7fe898141490 2024-04-02T16:11:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.232+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe898068d00 con 0x7fe898141490 2024-04-02T16:11:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.236+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8880be070 con 0x7fe898141490 2024-04-02T16:11:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.236+0000 7fe88effd640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 0x7fe87c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.236+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe88811fcf0 con 0x7fe898141490 2024-04-02T16:11:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.236+0000 7fe89e34b640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 0x7fe87c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.236+0000 7fe89e34b640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 0x7fe87c06a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8900043f0 tx=0x7fe89000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.240+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fe8880167c0 con 0x7fe898141490 2024-04-02T16:11:42.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe898144770 con 0x7fe898141490 2024-04-02T16:11:42.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe88effd640 1 -- 172.21.15.40:0/2827974313 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe8880efcf0 con 0x7fe898141490 2024-04-02T16:11:42.437 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:42.438 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 msgr2=0x7fe87c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 0x7fe87c06a840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe8900043f0 tx=0x7fe89000b040 comp rx=0 tx=0).stop 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 msgr2=0x7fe898141910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.432+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe888009b10 tx=0x7fe884005840 comp rx=0 tx=0).stop 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 shutdown_connections 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe87c068380 0x7fe87c06a840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe898141490 0x7fe898141910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe89813ffc0 0x7fe898140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 --2- 172.21.15.40:0/2827974313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe884004d50 0x7fe8981489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 >> 172.21.15.40:0/2827974313 conn(0x7fe8980b7f60 msgr2=0x7fe8980719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 shutdown_connections 2024-04-02T16:11:42.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.436+0000 7fe8a05af640 1 -- 172.21.15.40:0/2827974313 wait complete. 2024-04-02T16:11:42.455 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-02T16:11:42.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 Processor -- start 2024-04-02T16:11:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 -- start start 2024-04-02T16:11:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa2a80b4bf0 0x7fa2a80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2a80b8c80 0x7fa2a80b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a80b6310 con 0x7fa2a80b3500 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a80b55d0 con 0x7fa2a80b8c80 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2ad8f1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a814b560 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2a80b8c80 0x7fa2a80b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa2a80b8c80 0x7fa2a80b90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37562/0 (socket says 172.21.15.40:37562) 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1063241974 learned_addr learned my addr 172.21.15.40:0/1063241974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a6ffd640 1 --2- 172.21.15.40:0/1063241974 >> v2:172.21.15.204:3300/0 conn(0x7fa2a80b4bf0 0x7fa2a80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1063241974 >> v2:172.21.15.204:3300/0 conn(0x7fa2a80b4bf0 msgr2=0x7fa2a80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 --2- 172.21.15.40:0/1063241974 >> v2:172.21.15.204:3300/0 conn(0x7fa2a80b4bf0 0x7fa2a80b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1063241974 >> v1:172.21.15.204:6789/0 conn(0x7fa2a80b3500 legacy=0x7fa2a814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.560+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1063241974 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a80b3fa0 con 0x7fa2a80b8c80 2024-04-02T16:11:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a77fe640 1 --2- 172.21.15.40:0/1063241974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b8c80 0x7fa2a80b90b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80bab20 tx=0x7fa2a001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50419a2ac513d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a5ffb640 1 -- 172.21.15.40:0/1063241974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00b0070 con 0x7fa2a80b8c80 2024-04-02T16:11:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a5ffb640 1 -- 172.21.15.40:0/1063241974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a00ab600 con 0x7fa2a80b8c80 2024-04-02T16:11:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a5ffb640 1 -- 172.21.15.40:0/1063241974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00ab9d0 con 0x7fa2a80b8c80 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/1063241974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b8c80 msgr2=0x7fa2a80b90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/1063241974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b8c80 0x7fa2a80b90b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80bab20 tx=0x7fa2a001c400 comp rx=0 tx=0).stop 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/1063241974 shutdown_connections 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/1063241974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b8c80 0x7fa2a80b90b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/1063241974 >> v2:172.21.15.204:3300/0 conn(0x7fa2a80b4bf0 0x7fa2a80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/1063241974 >> 172.21.15.40:0/1063241974 conn(0x7fa2a80734b0 msgr2=0x7fa2a80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/1063241974 shutdown_connections 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/1063241974 wait complete. 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 Processor -- start 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- start start 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 0x7fa2a8147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a80b8c80 0x7fa2a81513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8154d90 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8154c10 con 0x7fa2a80b3500 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a8154f10 con 0x7fa2a80b8c80 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37572/0 (socket says 172.21.15.40:37572) 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 -- 172.21.15.40:0/147611965 learned_addr learned my addr 172.21.15.40:0/147611965 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a77fe640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a80b8c80 0x7fa2a81513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a6ffd640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 0x7fa2a8147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 -- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a80b8c80 msgr2=0x7fa2a81513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a80b8c80 0x7fa2a81513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 -- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 msgr2=0x7fa2a8147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 0x7fa2a8147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 -- 172.21.15.40:0/147611965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a814cc10 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a6ffd640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 0x7fa2a8147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:42.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2a67fc640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c420 tx=0x7fa29000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900052d0 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa290005470 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a8147f30 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290007260 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.564+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81e3780 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a80685f0 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa290007400 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa29ffff640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 0x7fa28806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa290085e90 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa2a6ffd640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 0x7fa28806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.568+0000 7fa2a6ffd640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 0x7fa28806a870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa298007330 tx=0x7fa29800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.572+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa290056440 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.728+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2a80bbd10 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.728+0000 7fa29ffff640 1 -- 172.21.15.40:0/147611965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fa290087090 con 0x7fa2a80b4bf0 2024-04-02T16:11:42.734 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:42.734 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 msgr2=0x7fa28806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 0x7fa28806a870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa298007330 tx=0x7fa29800b040 comp rx=0 tx=0).stop 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 msgr2=0x7fa2a8143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c420 tx=0x7fa29000c8e0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 shutdown_connections 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa2a80b8c80 0x7fa2a81513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa2880683b0 0x7fa28806a870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a80b4bf0 0x7fa2a8143460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 --2- 172.21.15.40:0/147611965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2a80b3500 0x7fa2a8147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 >> 172.21.15.40:0/147611965 conn(0x7fa2a80734b0 msgr2=0x7fa2a80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 shutdown_connections 2024-04-02T16:11:42.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.732+0000 7fa2ad8f1640 1 -- 172.21.15.40:0/147611965 wait complete. 2024-04-02T16:11:42.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:42.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.872+0000 7fc9f4997640 1 Processor -- start 2024-04-02T16:11:42.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.872+0000 7fc9f4997640 1 -- start start 2024-04-02T16:11:42.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9f4997640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc9f0145550 0x7fc9f01459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9f4997640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f00b2e40 con 0x7fc9f00718d0 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9f4997640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f00b42c0 con 0x7fc9f00b1d40 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9f4997640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f00b3e30 con 0x7fc9f0145550 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9eed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc9f00718d0 0x7fc9f0071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50964/0 (socket says 172.21.15.40:50964) 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9eed9d640 1 -- 172.21.15.40:0/3428901265 learned_addr learned my addr 172.21.15.40:0/3428901265 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9edd9b640 1 --2- 172.21.15.40:0/3428901265 >> v2:172.21.15.204:3300/0 conn(0x7fc9f0145550 0x7fc9f01459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467640441 0 0) 0x7fc9f00b2e40 con 0x7fc9f00718d0 2024-04-02T16:11:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9dc003530 con 0x7fc9f00718d0 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 106925475 0 0) 0x7fc9dc003530 con 0x7fc9f00718d0 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 >> v2:172.21.15.204:3300/0 conn(0x7fc9f0145550 msgr2=0x7fc9f01459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 --2- 172.21.15.40:0/3428901265 >> v2:172.21.15.204:3300/0 conn(0x7fc9f0145550 0x7fc9f01459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 >> v1:172.21.15.204:6789/0 conn(0x7fc9f00b1d40 legacy=0x7fc9f0072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f00b3820 con 0x7fc9f00718d0 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fc9d8001680 con 0x7fc9f00718d0 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 >> v1:172.21.15.40:6789/0 conn(0x7fc9f00718d0 legacy=0x7fc9f0071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 0x7fc9dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0043c0 0x7fc9dc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9dc005090 0x7fc9dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00b3e30 con 0x7fc9dc0039c0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc00bff0 con 0x7fc9dc005090 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc9dc00c1f0 con 0x7fc9dc0043c0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 0x7fc9dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9eed9d640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9dc005090 0x7fc9dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9edd9b640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0043c0 0x7fc9dc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 -- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0043c0 msgr2=0x7fc9dc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0043c0 0x7fc9dc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 -- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9dc005090 msgr2=0x7fc9dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9dc005090 0x7fc9dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9ee59c640 1 -- 172.21.15.40:0/3428901265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc003530 con 0x7fc9dc0039c0 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9eed9d640 1 -- 172.21.15.40:0/3428901265 reap_dead start 2024-04-02T16:11:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.876+0000 7fc9edd9b640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0043c0 0x7fc9dc004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 0x7fc9dc003da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0001860 tx=0x7fc9e001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d563b1b9b5fd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00b0070 con 0x7fc9dc0039c0 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e00ab600 con 0x7fc9dc0039c0 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9ed59a640 1 -- 172.21.15.40:0/3428901265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0001460 con 0x7fc9dc0039c0 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 msgr2=0x7fc9dc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 0x7fc9dc003da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0001860 tx=0x7fc9e001c500 comp rx=0 tx=0).stop 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/3428901265 shutdown_connections 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 --2- 172.21.15.40:0/3428901265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9dc0039c0 0x7fc9dc003da0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/3428901265 >> 172.21.15.40:0/3428901265 conn(0x7fc9f00b7d10 msgr2=0x7fc9f00b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/3428901265 shutdown_connections 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/3428901265 wait complete. 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 Processor -- start 2024-04-02T16:11:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- start start 2024-04-02T16:11:42.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0039c0 0x7fc9f006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f006dbd0 0x7fc9f013a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0072040 con 0x7fc9f006e050 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0071ec0 con 0x7fc9f006dbd0 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f00721c0 con 0x7fc9dc0039c0 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37582/0 (socket says 172.21.15.40:37582) 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 -- 172.21.15.40:0/2346900541 learned_addr learned my addr 172.21.15.40:0/2346900541 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0039c0 0x7fc9f006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9edd9b640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f006dbd0 0x7fc9f013a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 -- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0039c0 msgr2=0x7fc9f006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0039c0 0x7fc9f006bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 -- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f006dbd0 msgr2=0x7fc9f013a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f006dbd0 0x7fc9f013a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 -- 172.21.15.40:0/2346900541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f00b0040 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9eed9d640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0145ee0 tx=0x7fc9d800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80105d0 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8010790 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f006d470 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8010b80 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.880+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f0147d90 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f006c520 con 0x7fc9f006e050 2024-04-02T16:11:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc9d8014430 con 0x7fc9f006e050 2024-04-02T16:11:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9e6ffd640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 0x7fc9d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc9d8086060 con 0x7fc9f006e050 2024-04-02T16:11:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 0x7fc9d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.884+0000 7fc9ee59c640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 0x7fc9d006a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00ad000 tx=0x7fc9e00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:42.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:42.888+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc9d80566b0 con 0x7fc9f006e050 2024-04-02T16:11:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.052+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9f0148170 con 0x7fc9f006e050 2024-04-02T16:11:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.052+0000 7fc9e6ffd640 1 -- 172.21.15.40:0/2346900541 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc9f0148170 con 0x7fc9f006e050 2024-04-02T16:11:43.058 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 msgr2=0x7fc9d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 0x7fc9d006a840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00ad000 tx=0x7fc9e00a6000 comp rx=0 tx=0).stop 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 msgr2=0x7fc9f013a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0145ee0 tx=0x7fc9d800e960 comp rx=0 tx=0).stop 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 shutdown_connections 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc9d0068380 0x7fc9d006a840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9f006e050 0x7fc9f013a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9f006dbd0 0x7fc9f013a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 --2- 172.21.15.40:0/2346900541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9dc0039c0 0x7fc9f006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 >> 172.21.15.40:0/2346900541 conn(0x7fc9f00b7d10 msgr2=0x7fc9f00b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 shutdown_connections 2024-04-02T16:11:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.056+0000 7fc9f4997640 1 -- 172.21.15.40:0/2346900541 wait complete. 2024-04-02T16:11:43.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 Processor -- start 2024-04-02T16:11:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 -- start start 2024-04-02T16:11:43.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96f006c8a0 0x7f96f00af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96f00b3800 0x7f96f00b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b41d0 0x7f96f0147c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96f00b2b50 con 0x7f96f00b3800 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f96f00b0300 con 0x7f96f006c8a0 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f7709640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96f00b08c0 con 0x7f96f00b41d0 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96f00b3800 0x7f96f00b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96f00b3800 0x7f96f00b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37596/0 (socket says 172.21.15.40:37596) 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 -- 172.21.15.40:0/2422922813 learned_addr learned my addr 172.21.15.40:0/2422922813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f54a5640 1 --2- 172.21.15.40:0/2422922813 >> v2:172.21.15.204:3300/0 conn(0x7f96f006c8a0 0x7f96f00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f5ca6640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b41d0 0x7f96f0147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 -- 172.21.15.40:0/2422922813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b41d0 msgr2=0x7f96f0147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b41d0 0x7f96f0147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 -- 172.21.15.40:0/2422922813 >> v2:172.21.15.204:3300/0 conn(0x7f96f006c8a0 msgr2=0x7f96f00af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 --2- 172.21.15.40:0/2422922813 >> v2:172.21.15.204:3300/0 conn(0x7f96f006c8a0 0x7f96f00af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.192+0000 7f96f4ca4640 1 -- 172.21.15.40:0/2422922813 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f00ae830 con 0x7f96f00b3800 2024-04-02T16:11:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f4ca4640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b3800 0x7f96f00b3c90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f96f006e070 tx=0x7f96e001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abfa8172e6baeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e7fff640 1 -- 172.21.15.40:0/2422922813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00b0070 con 0x7f96f00b3800 2024-04-02T16:11:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e7fff640 1 -- 172.21.15.40:0/2422922813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e00ab600 con 0x7f96f00b3800 2024-04-02T16:11:43.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e7fff640 1 -- 172.21.15.40:0/2422922813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00ab9d0 con 0x7f96f00b3800 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2422922813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b3800 msgr2=0x7f96f00b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b3800 0x7f96f00b3c90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f96f006e070 tx=0x7f96e001c140 comp rx=0 tx=0).stop 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2422922813 shutdown_connections 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b41d0 0x7f96f0147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- 172.21.15.40:0/2422922813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b3800 0x7f96f00b3c90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- 172.21.15.40:0/2422922813 >> v2:172.21.15.204:3300/0 conn(0x7f96f006c8a0 0x7f96f00af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2422922813 >> 172.21.15.40:0/2422922813 conn(0x7f96f00b8fd0 msgr2=0x7f96f00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2422922813 shutdown_connections 2024-04-02T16:11:43.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2422922813 wait complete. 2024-04-02T16:11:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 Processor -- start 2024-04-02T16:11:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- start start 2024-04-02T16:11:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 0x7f96f013a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b3800 0x7f96f01de390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00af690 con 0x7f96f00b41d0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00b0540 con 0x7f96f006c8a0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96f00b3130 con 0x7f96f00b3800 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37606/0 (socket says 172.21.15.40:37606) 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 -- 172.21.15.40:0/2559986771 learned_addr learned my addr 172.21.15.40:0/2559986771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f4ca4640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b3800 0x7f96f01de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f54a5640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 0x7f96f013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 -- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b3800 msgr2=0x7f96f01de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b3800 0x7f96f01de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 -- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 msgr2=0x7f96f013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 0x7f96f013a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 -- 172.21.15.40:0/2559986771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f0138ea0 con 0x7f96f00b41d0 2024-04-02T16:11:43.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f54a5640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 0x7f96f013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f5ca6640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007e90 tx=0x7f96e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8014420 con 0x7f96f00b41d0 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e8019070 con 0x7f96f00b41d0 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f013def0 con 0x7f96f00b41d0 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8014800 con 0x7f96f00b41d0 2024-04-02T16:11:43.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.196+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f0141840 con 0x7f96f00b41d0 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f00685f0 con 0x7f96f00b41d0 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f96e80149a0 con 0x7f96f00b41d0 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96e5ffb640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 0x7f96d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f96e80870e0 con 0x7f96f00b41d0 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96f54a5640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 0x7f96d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.200+0000 7f96f54a5640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 0x7f96d406a840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009840 tx=0x7f96ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.204+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f96e8057730 con 0x7f96f00b41d0 2024-04-02T16:11:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.364+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96f00b1d40 con 0x7f96f00b41d0 2024-04-02T16:11:43.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96e5ffb640 1 -- 172.21.15.40:0/2559986771 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f96e8057030 con 0x7f96f00b41d0 2024-04-02T16:11:43.372 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 msgr2=0x7f96d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 0x7f96d406a840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009840 tx=0x7f96ec008040 comp rx=0 tx=0).stop 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 msgr2=0x7f96f009ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007e90 tx=0x7f96e800e4b0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 shutdown_connections 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f00b3800 0x7f96f01de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f96d4068380 0x7f96d406a840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96f00b41d0 0x7f96f009ca90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 --2- 172.21.15.40:0/2559986771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f006c8a0 0x7f96f013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 >> 172.21.15.40:0/2559986771 conn(0x7f96f00b8fd0 msgr2=0x7f96f00aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 shutdown_connections 2024-04-02T16:11:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.368+0000 7f96f7709640 1 -- 172.21.15.40:0/2559986771 wait complete. 2024-04-02T16:11:43.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:43.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.512+0000 7f2af0784640 1 Processor -- start 2024-04-02T16:11:43.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.512+0000 7f2af0784640 1 -- start start 2024-04-02T16:11:43.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2af0784640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ae8073230 0x7f2ae80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2af0784640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80b19e0 con 0x7f2ae8148e50 2024-04-02T16:11:43.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2af0784640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80b05c0 con 0x7f2ae80b23d0 2024-04-02T16:11:43.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2af0784640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae80b08a0 con 0x7f2ae8073230 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aedd1f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ae8073230 0x7f2ae80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aeed21640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ae8148e50 0x7f2ae8149280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50974/0 (socket says 172.21.15.40:50974) 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aeed21640 1 -- 172.21.15.40:0/4077622464 learned_addr learned my addr 172.21.15.40:0/4077622464 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aedd1f640 1 -- 172.21.15.40:0/4077622464 >> v1:172.21.15.204:6789/0 conn(0x7f2ae80b23d0 legacy=0x7f2ae809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aedd1f640 1 -- 172.21.15.40:0/4077622464 >> v1:172.21.15.40:6789/0 conn(0x7f2ae8148e50 legacy=0x7f2ae8149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aedd1f640 1 -- 172.21.15.40:0/4077622464 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80b0b80 con 0x7f2ae8073230 2024-04-02T16:11:43.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aedd1f640 1 --2- 172.21.15.40:0/4077622464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8073230 0x7f2ae80b27b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097c0 tx=0x7f2adc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb0a95ba7bdd64f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aed51e640 1 -- 172.21.15.40:0/4077622464 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0b0070 con 0x7f2ae8073230 2024-04-02T16:11:43.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aed51e640 1 -- 172.21.15.40:0/4077622464 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc0ab600 con 0x7f2ae8073230 2024-04-02T16:11:43.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.516+0000 7f2aed51e640 1 -- 172.21.15.40:0/4077622464 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0ab9d0 con 0x7f2ae8073230 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/4077622464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8073230 msgr2=0x7f2ae80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 --2- 172.21.15.40:0/4077622464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8073230 0x7f2ae80b27b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097c0 tx=0x7f2adc01c410 comp rx=0 tx=0).stop 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/4077622464 shutdown_connections 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 --2- 172.21.15.40:0/4077622464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae8073230 0x7f2ae80b27b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/4077622464 >> 172.21.15.40:0/4077622464 conn(0x7f2ae80b8fd0 msgr2=0x7f2ae80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/4077622464 shutdown_connections 2024-04-02T16:11:43.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/4077622464 wait complete. 2024-04-02T16:11:43.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 Processor -- start 2024-04-02T16:11:43.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- start start 2024-04-02T16:11:43.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ae8073230 0x7f2ae806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae8148e50 0x7f2ae81de470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae8153a30 con 0x7f2ae80b23d0 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae81538b0 con 0x7f2ae8148e50 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae8153bb0 con 0x7f2ae8073230 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37632/0 (socket says 172.21.15.40:37632) 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 -- 172.21.15.40:0/2347080533 learned_addr learned my addr 172.21.15.40:0/2347080533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 -- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ae8073230 msgr2=0x7f2ae806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ae8073230 0x7f2ae806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 -- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae8148e50 msgr2=0x7f2ae81de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae8148e50 0x7f2ae81de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 -- 172.21.15.40:0/2347080533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae8070e20 con 0x7f2ae80b23d0 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2aedd1f640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0ae040 tx=0x7f2adc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0b8020 con 0x7f2ae80b23d0 2024-04-02T16:11:43.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc0b4630 con 0x7f2ae80b23d0 2024-04-02T16:11:43.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae8071100 con 0x7f2ae80b23d0 2024-04-02T16:11:43.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0b0040 con 0x7f2ae80b23d0 2024-04-02T16:11:43.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae813ac20 con 0x7f2ae80b23d0 2024-04-02T16:11:43.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.520+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae80685f0 con 0x7f2ae80b23d0 2024-04-02T16:11:43.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.524+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2adc0bb020 con 0x7f2ae80b23d0 2024-04-02T16:11:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.524+0000 7f2adaffd640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 0x7f2acc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.524+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2adc120670 con 0x7f2ae80b23d0 2024-04-02T16:11:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.524+0000 7f2aee520640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 0x7f2acc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.524+0000 7f2aee520640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 0x7f2acc06a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f70 tx=0x7f2ae40079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.528+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2adc0c2030 con 0x7f2ae80b23d0 2024-04-02T16:11:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ae80b0300 con 0x7f2ae80b23d0 2024-04-02T16:11:43.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2adaffd640 1 -- 172.21.15.40:0/2347080533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2adc016090 con 0x7f2ae80b23d0 2024-04-02T16:11:43.709 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 msgr2=0x7f2acc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 0x7f2acc06a840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f70 tx=0x7f2ae40079e0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 msgr2=0x7f2ae81ddf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0ae040 tx=0x7f2adc0191b0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 shutdown_connections 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2acc068380 0x7f2acc06a840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ae8148e50 0x7f2ae81de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ae80b23d0 0x7f2ae81ddf30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 --2- 172.21.15.40:0/2347080533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ae8073230 0x7f2ae806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 >> 172.21.15.40:0/2347080533 conn(0x7f2ae80b8fd0 msgr2=0x7f2ae80b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 shutdown_connections 2024-04-02T16:11:43.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.704+0000 7f2af0784640 1 -- 172.21.15.40:0/2347080533 wait complete. 2024-04-02T16:11:43.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:43.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 Processor -- start 2024-04-02T16:11:43.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 -- start start 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2710073230 0x7f27100b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710148e50 0x7f2710149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27100b19e0 con 0x7f27100b23d0 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f27101497c0 con 0x7f2710073230 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.840+0000 7f2714903640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27100b05c0 con 0x7f2710148e50 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2710073230 0x7f27100b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2710073230 0x7f27100b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37640/0 (socket says 172.21.15.40:37640) 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 -- 172.21.15.40:0/2997181368 learned_addr learned my addr 172.21.15.40:0/2997181368 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:43.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270ed9d640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710148e50 0x7f2710149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 -- 172.21.15.40:0/2997181368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710148e50 msgr2=0x7f2710149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:43.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710148e50 0x7f2710149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 -- 172.21.15.40:0/2997181368 >> v1:172.21.15.204:6789/0 conn(0x7f27100b23d0 legacy=0x7f271009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 -- 172.21.15.40:0/2997181368 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27100b08a0 con 0x7f2710073230 2024-04-02T16:11:43.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270dd9b640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710073230 0x7f27100b27b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2704009870 tx=0x7f270401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a44a5c5dc47d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270d59a640 1 -- 172.21.15.40:0/2997181368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040aa650 con 0x7f2710073230 2024-04-02T16:11:43.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270d59a640 1 -- 172.21.15.40:0/2997181368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27040af070 con 0x7f2710073230 2024-04-02T16:11:43.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f270d59a640 1 -- 172.21.15.40:0/2997181368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27040aaa10 con 0x7f2710073230 2024-04-02T16:11:43.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 -- 172.21.15.40:0/2997181368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710073230 msgr2=0x7f27100b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710073230 0x7f27100b27b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2704009870 tx=0x7f270401c4d0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 -- 172.21.15.40:0/2997181368 shutdown_connections 2024-04-02T16:11:43.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710148e50 0x7f2710149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 --2- 172.21.15.40:0/2997181368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710073230 0x7f27100b27b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 -- 172.21.15.40:0/2997181368 >> 172.21.15.40:0/2997181368 conn(0x7f27100b8fd0 msgr2=0x7f27100b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:43.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 -- 172.21.15.40:0/2997181368 shutdown_connections 2024-04-02T16:11:43.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.844+0000 7f2714903640 1 -- 172.21.15.40:0/2997181368 wait complete. 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 Processor -- start 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- start start 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710073230 0x7f2710142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27100b23d0 0x7f27101417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2710152810 con 0x7f2710148e50 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2710152690 con 0x7f27100b23d0 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2710152990 con 0x7f2710073230 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37646/0 (socket says 172.21.15.40:37646) 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 -- 172.21.15.40:0/4199089023 learned_addr learned my addr 172.21.15.40:0/4199089023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270e59c640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710073230 0x7f2710142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270dd9b640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27100b23d0 0x7f27101417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 -- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710073230 msgr2=0x7f2710142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710073230 0x7f2710142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 -- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27100b23d0 msgr2=0x7f27101417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27100b23d0 0x7f27101417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 -- 172.21.15.40:0/4199089023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101400d0 con 0x7f2710148e50 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f270ed9d640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f270800d590 tx=0x7f270800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080125f0 con 0x7f2710148e50 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2708019070 con 0x7f2710148e50 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2710141cf0 con 0x7f2710148e50 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080129f0 con 0x7f2710148e50 2024-04-02T16:11:43.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101461f0 con 0x7f2710148e50 2024-04-02T16:11:43.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27100685f0 con 0x7f2710148e50 2024-04-02T16:11:43.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2708012b90 con 0x7f2710148e50 2024-04-02T16:11:43.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 0x7f26f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:43.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.848+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2708086d30 con 0x7f2710148e50 2024-04-02T16:11:43.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.852+0000 7f270e59c640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 0x7f26f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:43.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.852+0000 7f270e59c640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 0x7f26f006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2700001cc0 tx=0x7f2700009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:43.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:43.856+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2708057380 con 0x7f2710148e50 2024-04-02T16:11:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27100b1690 con 0x7f2710148e50 2024-04-02T16:11:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f26feffd640 1 -- 172.21.15.40:0/4199089023 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7f2708056c80 con 0x7f2710148e50 2024-04-02T16:11:44.049 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:44.050 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:11:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:11:44.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 msgr2=0x7f26f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 0x7f26f006a840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2700001cc0 tx=0x7f2700009040 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 msgr2=0x7f2710140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f270800d590 tx=0x7f270800da50 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 shutdown_connections 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26f0068380 0x7f26f006a840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2710148e50 0x7f2710140890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27100b23d0 0x7f27101417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 --2- 172.21.15.40:0/4199089023 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2710073230 0x7f2710142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 >> 172.21.15.40:0/4199089023 conn(0x7f27100b8fd0 msgr2=0x7f271006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 shutdown_connections 2024-04-02T16:11:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.044+0000 7f2714903640 1 -- 172.21.15.40:0/4199089023 wait complete. 2024-04-02T16:11:44.067 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:44.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 Processor -- start 2024-04-02T16:11:44.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 -- start start 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36dc0b6310 con 0x7f36dc0b8c80 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36dc0b55d0 con 0x7f36dc0b3500 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36e09b1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f36dc14b560 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37656/0 (socket says 172.21.15.40:37656) 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 -- 172.21.15.40:0/106636825 learned_addr learned my addr 172.21.15.40:0/106636825 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 -- 172.21.15.40:0/106636825 >> v1:172.21.15.204:6789/0 conn(0x7f36dc0b3500 legacy=0x7f36dc14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 -- 172.21.15.40:0/106636825 >> v1:172.21.15.40:6789/0 conn(0x7f36dc0b8c80 legacy=0x7f36dc1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.172+0000 7f36da59c640 1 -- 172.21.15.40:0/106636825 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc0b3fa0 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- 172.21.15.40:0/106636825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009870 tx=0x7f36cc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d96a52a0445a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36d959a640 1 -- 172.21.15.40:0/106636825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0b0070 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36d959a640 1 -- 172.21.15.40:0/106636825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36cc0ab600 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36d959a640 1 -- 172.21.15.40:0/106636825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0ab9d0 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/106636825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b4bf0 msgr2=0x7f36dc0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 --2- 172.21.15.40:0/106636825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009870 tx=0x7f36cc01c400 comp rx=0 tx=0).stop 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/106636825 shutdown_connections 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 --2- 172.21.15.40:0/106636825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b4bf0 0x7f36dc0b4fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/106636825 >> 172.21.15.40:0/106636825 conn(0x7f36dc0734b0 msgr2=0x7f36dc0bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/106636825 shutdown_connections 2024-04-02T16:11:44.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/106636825 wait complete. 2024-04-02T16:11:44.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 Processor -- start 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- start start 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36dc0b4bf0 0x7f36dc144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 0x7f36dc146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc156740 con 0x7f36dc0b3500 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1568c0 con 0x7f36dc0b8c80 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36dc156a40 con 0x7f36dc0b4bf0 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37660/0 (socket says 172.21.15.40:37660) 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 -- 172.21.15.40:0/1624303064 learned_addr learned my addr 172.21.15.40:0/1624303064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36dad9d640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 0x7f36dc146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36d9d9b640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36dc0b4bf0 0x7f36dc144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 -- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36dc0b4bf0 msgr2=0x7f36dc144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36dc0b4bf0 0x7f36dc144710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 -- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 msgr2=0x7f36dc146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 0x7f36dc146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 -- 172.21.15.40:0/1624303064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc144340 con 0x7f36dc0b3500 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36dad9d640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 0x7f36dc146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36da59c640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0ae040 tx=0x7f36cc0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0b0040 con 0x7f36dc0b3500 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36cc0aa730 con 0x7f36dc0b3500 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc144060 con 0x7f36dc0b3500 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0aab40 con 0x7f36dc0b3500 2024-04-02T16:11:44.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.176+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc143060 con 0x7f36dc0b3500 2024-04-02T16:11:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36dc145110 con 0x7f36dc0b3500 2024-04-02T16:11:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f36cc0bb020 con 0x7f36dc0b3500 2024-04-02T16:11:44.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36caffd640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 0x7f36bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36cc120290 con 0x7f36dc0b3500 2024-04-02T16:11:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36d9d9b640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 0x7f36bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.180+0000 7f36d9d9b640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 0x7f36bc06a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0b3e40 tx=0x7f36d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.184+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f36dc145110 con 0x7f36dc0b3500 2024-04-02T16:11:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.348+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36dc0685f0 con 0x7f36dc0b3500 2024-04-02T16:11:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36caffd640 1 -- 172.21.15.40:0/1624303064 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f36cc11d0c0 con 0x7f36dc0b3500 2024-04-02T16:11:44.357 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:44.357 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 msgr2=0x7f36bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 0x7f36bc06a840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0b3e40 tx=0x7f36d0009040 comp rx=0 tx=0).stop 2024-04-02T16:11:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 msgr2=0x7f36dc1439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0ae040 tx=0x7f36cc0191a0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 shutdown_connections 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36dc0b4bf0 0x7f36dc144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36bc068380 0x7f36bc06a840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36dc0b8c80 0x7f36dc146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 --2- 172.21.15.40:0/1624303064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36dc0b3500 0x7f36dc1439c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 >> 172.21.15.40:0/1624303064 conn(0x7f36dc0734b0 msgr2=0x7f36dc0baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 shutdown_connections 2024-04-02T16:11:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.352+0000 7f36e09b1640 1 -- 172.21.15.40:0/1624303064 wait complete. 2024-04-02T16:11:44.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:44.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:44.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 Processor -- start 2024-04-02T16:11:44.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 -- start start 2024-04-02T16:11:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f75d00b45a0 0x7f75d00b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d009bef0 con 0x7f75d0144a30 2024-04-02T16:11:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d002c840 con 0x7f75d002c460 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d7064640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f75d0145570 con 0x7f75d00b45a0 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d5601640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f75d0144a30 0x7f75d0144e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50980/0 (socket says 172.21.15.40:50980) 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d5601640 1 -- 172.21.15.40:0/3076263430 learned_addr learned my addr 172.21.15.40:0/3076263430 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cffff640 1 --2- 172.21.15.40:0/3076263430 >> v2:172.21.15.204:3300/0 conn(0x7f75d00b45a0 0x7f75d00b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2287115319 0 0) 0x7f75d009bef0 con 0x7f75d0144a30 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75bc003530 con 0x7f75d0144a30 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3469464247 0 0) 0x7f75bc003530 con 0x7f75d0144a30 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 >> v2:172.21.15.204:3300/0 conn(0x7f75d00b45a0 msgr2=0x7f75d00b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 --2- 172.21.15.40:0/3076263430 >> v2:172.21.15.204:3300/0 conn(0x7f75d00b45a0 0x7f75d00b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 >> v1:172.21.15.204:6789/0 conn(0x7f75d002c460 legacy=0x7f75d0147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d00aef40 con 0x7f75d0144a30 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f75c00016a0 con 0x7f75d0144a30 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 >> v1:172.21.15.40:6789/0 conn(0x7f75d0144a30 legacy=0x7f75d0144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75bc0052f0 0x7f75bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0145570 con 0x7f75bc0039c0 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc00c1c0 con 0x7f75bc004810 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75bc00c3c0 con 0x7f75bc0052f0 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cffff640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d5601640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75bc0052f0 0x7f75bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 -- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75bc0052f0 msgr2=0x7f75bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75bc0052f0 0x7f75bc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 -- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc004810 msgr2=0x7f75bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d4e00640 1 -- 172.21.15.40:0/3076263430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc003530 con 0x7f75bc0039c0 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75cffff640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc004810 0x7f75bc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.496+0000 7f75d5601640 1 -- 172.21.15.40:0/3076263430 reap_dead start 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d4e00640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75c8001e80 tx=0x7f75c801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd403fa91529f968 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80b0070 con 0x7f75bc0039c0 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8003bd0 con 0x7f75bc0039c0 2024-04-02T16:11:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cf7fe640 1 -- 172.21.15.40:0/3076263430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8001ad0 con 0x7f75bc0039c0 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 msgr2=0x7f75bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75c8001e80 tx=0x7f75c801c700 comp rx=0 tx=0).stop 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- 172.21.15.40:0/3076263430 shutdown_connections 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 --2- 172.21.15.40:0/3076263430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc0039c0 0x7f75bc003de0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- 172.21.15.40:0/3076263430 >> 172.21.15.40:0/3076263430 conn(0x7f75d00b5fe0 msgr2=0x7f75d00b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- 172.21.15.40:0/3076263430 shutdown_connections 2024-04-02T16:11:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- 172.21.15.40:0/3076263430 wait complete. 2024-04-02T16:11:44.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 Processor -- start 2024-04-02T16:11:44.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- start start 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc0039c0 0x7f75d00b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75d00b2910 0x7f75d0142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00aec60 con 0x7f75d0145b50 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00aea10 con 0x7f75bc0039c0 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d7064640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75d0144e40 con 0x7f75d00b2910 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37682/0 (socket says 172.21.15.40:37682) 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 -- 172.21.15.40:0/4112714831 learned_addr learned my addr 172.21.15.40:0/4112714831 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d5601640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75d00b2910 0x7f75d0142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75d4e00640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc0039c0 0x7f75d00b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 -- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75d00b2910 msgr2=0x7f75d0142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75d00b2910 0x7f75d0142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 -- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc0039c0 msgr2=0x7f75d00b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc0039c0 0x7f75d00b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 -- 172.21.15.40:0/4112714831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d00b2270 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.500+0000 7f75cffff640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75b800e810 tx=0x7f75b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8017070 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b800c8d0 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d009c750 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b800f040 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d00b1530 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00685f0 con 0x7f75d0145b50 2024-04-02T16:11:44.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f75b801a070 con 0x7f75d0145b50 2024-04-02T16:11:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 0x7f75b006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75b8086490 con 0x7f75d0145b50 2024-04-02T16:11:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75d4e00640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 0x7f75b006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.504+0000 7f75d4e00640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 0x7f75b006a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75c80ad000 tx=0x7f75c8001570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.508+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f75b8056ac0 con 0x7f75d0145b50 2024-04-02T16:11:44.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75d00b8aa0 con 0x7f75d0145b50 2024-04-02T16:11:44.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75cd7fa640 1 -- 172.21.15.40:0/4112714831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f75b80563c0 con 0x7f75d0145b50 2024-04-02T16:11:44.677 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:44.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 msgr2=0x7f75b006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 0x7f75b006a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75c80ad000 tx=0x7f75c8001570 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 msgr2=0x7f75d00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75b800e810 tx=0x7f75b800ecd0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 shutdown_connections 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75d00b2910 0x7f75d0142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f75b00683b0 0x7f75b006a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75d0145b50 0x7f75d00b1d30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 --2- 172.21.15.40:0/4112714831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75bc0039c0 0x7f75d00b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 >> 172.21.15.40:0/4112714831 conn(0x7f75d00b5fe0 msgr2=0x7f75d01495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 shutdown_connections 2024-04-02T16:11:44.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.672+0000 7f75d7064640 1 -- 172.21.15.40:0/4112714831 wait complete. 2024-04-02T16:11:44.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:44.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 Processor -- start 2024-04-02T16:11:44.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 -- start start 2024-04-02T16:11:44.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69d80ae080 0x7f69d80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b1c10 0x7f69d80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69d80b3210 con 0x7f69d80b25b0 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f69d806db10 con 0x7f69d80ae080 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69dd1c4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f69d80ae460 con 0x7f69d80b1c10 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d6d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69d80ae080 0x7f69d80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d759e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f69d80b25b0 0x7f69d806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50988/0 (socket says 172.21.15.40:50988) 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d759e640 1 -- 172.21.15.40:0/4114020778 learned_addr learned my addr 172.21.15.40:0/4114020778 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d659c640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b1c10 0x7f69d80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d6d9d640 1 -- 172.21.15.40:0/4114020778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b1c10 msgr2=0x7f69d80b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d6d9d640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b1c10 0x7f69d80b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d6d9d640 1 -- 172.21.15.40:0/4114020778 >> v1:172.21.15.40:6789/0 conn(0x7f69d80b25b0 legacy=0x7f69d806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d6d9d640 1 -- 172.21.15.40:0/4114020778 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d80af670 con 0x7f69d80ae080 2024-04-02T16:11:44.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.816+0000 7f69d5d9b640 1 -- 172.21.15.40:0/4114020778 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471261283 0 0) 0x7f69d80b3210 con 0x7f69d80b25b0 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d80b0300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69c80097c0 tx=0x7f69c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba62b966e56c1926 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d5d9b640 1 -- 172.21.15.40:0/4114020778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c80b0070 con 0x7f69d80ae080 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d5d9b640 1 -- 172.21.15.40:0/4114020778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c80a4e50 con 0x7f69d80ae080 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d5d9b640 1 -- 172.21.15.40:0/4114020778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c80ab820 con 0x7f69d80ae080 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/4114020778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 msgr2=0x7f69d80b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d80b0300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f69c80097c0 tx=0x7f69c801c4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/4114020778 shutdown_connections 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b1c10 0x7f69d80b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/4114020778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d80b0300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/4114020778 >> 172.21.15.40:0/4114020778 conn(0x7f69d80b8fd0 msgr2=0x7f69d80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/4114020778 shutdown_connections 2024-04-02T16:11:44.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/4114020778 wait complete. 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 Processor -- start 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- start start 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69d80b1c10 0x7f69d809b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 0x7f69d814d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80aeeb0 con 0x7f69d80ae080 2024-04-02T16:11:44.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69d80af2f0 con 0x7f69d80b1c10 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f69d80af030 con 0x7f69d80b25b0 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37706/0 (socket says 172.21.15.40:37706) 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 -- 172.21.15.40:0/902488971 learned_addr learned my addr 172.21.15.40:0/902488971 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d659c640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69d80b1c10 0x7f69d809b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 -- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 msgr2=0x7f69d814d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d759e640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 0x7f69d814d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 0x7f69d814d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 -- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69d80b1c10 msgr2=0x7f69d809b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69d80b1c10 0x7f69d809b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 -- 172.21.15.40:0/902488971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d809d210 con 0x7f69d80ae080 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d759e640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 0x7f69d814d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:44.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69d6d9d640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69c80ae040 tx=0x7f69c801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c80b0050 con 0x7f69d80ae080 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c80b1040 con 0x7f69d80ae080 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d809d4f0 con 0x7f69d80ae080 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8006e20 con 0x7f69d80ae080 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.820+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d814e5c0 con 0x7f69d80ae080 2024-04-02T16:11:44.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d80685f0 con 0x7f69d80ae080 2024-04-02T16:11:44.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f69c80be060 con 0x7f69d80ae080 2024-04-02T16:11:44.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69cf7fe640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 0x7f69b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:44.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f69c8120140 con 0x7f69d80ae080 2024-04-02T16:11:44.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69d659c640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 0x7f69b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:44.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.824+0000 7f69d659c640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 0x7f69b806a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69d80b9760 tx=0x7f69c000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:44.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.828+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f69c8016090 con 0x7f69d80ae080 2024-04-02T16:11:44.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.984+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f69d80b8bb0 con 0x7f69d80ae080 2024-04-02T16:11:44.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.984+0000 7f69cf7fe640 1 -- 172.21.15.40:0/902488971 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f69c80f0790 con 0x7f69d80ae080 2024-04-02T16:11:44.990 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 msgr2=0x7f69b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 0x7f69b806a840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f69d80b9760 tx=0x7f69c000b040 comp rx=0 tx=0).stop 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 msgr2=0x7f69d8152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69c80ae040 tx=0x7f69c801ace0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 shutdown_connections 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69d80b25b0 0x7f69d814d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f69b8068380 0x7f69b806a840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69d80b1c10 0x7f69d809b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 --2- 172.21.15.40:0/902488971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69d80ae080 0x7f69d8152210 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:44.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 >> 172.21.15.40:0/902488971 conn(0x7f69d80b8fd0 msgr2=0x7f69d80b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:44.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 shutdown_connections 2024-04-02T16:11:44.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:44.988+0000 7f69dd1c4640 1 -- 172.21.15.40:0/902488971 wait complete. 2024-04-02T16:11:45.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.124+0000 7f0df0c09640 1 Processor -- start 2024-04-02T16:11:45.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.124+0000 7f0df0c09640 1 -- start start 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0df0c09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0dec09f0e0 0x7f0dec072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0df0c09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 0x7f0dec1520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0df0c09640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dec072040 con 0x7f0dec072ec0 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0df0c09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0dec141fb0 con 0x7f0dec09f0e0 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0df0c09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec1477f0 con 0x7f0dec148550 2024-04-02T16:11:45.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 0x7f0dec1520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0de3fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0dec072ec0 0x7f0dec147e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:49924/0 (socket says 172.21.15.40:49924) 2024-04-02T16:11:45.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0de3fff640 1 -- 172.21.15.40:0/38519923 learned_addr learned my addr 172.21.15.40:0/38519923 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:45.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dea59c640 1 --2- 172.21.15.40:0/38519923 >> v2:172.21.15.204:3300/0 conn(0x7f0dec09f0e0 0x7f0dec072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0de9d9b640 1 -- 172.21.15.40:0/38519923 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446445653 0 0) 0x7f0dec072040 con 0x7f0dec072ec0 2024-04-02T16:11:45.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0de9d9b640 1 -- 172.21.15.40:0/38519923 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dd8003530 con 0x7f0dec072ec0 2024-04-02T16:11:45.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 -- 172.21.15.40:0/38519923 >> v2:172.21.15.204:3300/0 conn(0x7f0dec09f0e0 msgr2=0x7f0dec072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:45.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 --2- 172.21.15.40:0/38519923 >> v2:172.21.15.204:3300/0 conn(0x7f0dec09f0e0 0x7f0dec072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 -- 172.21.15.40:0/38519923 >> v1:172.21.15.204:6789/0 conn(0x7f0dec072ec0 legacy=0x7f0dec147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 -- 172.21.15.40:0/38519923 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec070a60 con 0x7f0dec148550 2024-04-02T16:11:45.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.128+0000 7f0dead9d640 1 --2- 172.21.15.40:0/38519923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 0x7f0dec1520b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009d90 tx=0x7f0de401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80927c399feba48c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de9d9b640 1 -- 172.21.15.40:0/38519923 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40b2070 con 0x7f0dec148550 2024-04-02T16:11:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de9d9b640 1 -- 172.21.15.40:0/38519923 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de40a58b0 con 0x7f0dec148550 2024-04-02T16:11:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de9d9b640 1 -- 172.21.15.40:0/38519923 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de40a5c60 con 0x7f0dec148550 2024-04-02T16:11:45.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/38519923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 msgr2=0x7f0dec1520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- 172.21.15.40:0/38519923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 0x7f0dec1520b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009d90 tx=0x7f0de401c8e0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/38519923 shutdown_connections 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- 172.21.15.40:0/38519923 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec148550 0x7f0dec1520b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- 172.21.15.40:0/38519923 >> v2:172.21.15.204:3300/0 conn(0x7f0dec09f0e0 0x7f0dec072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/38519923 >> 172.21.15.40:0/38519923 conn(0x7f0dec0b7d10 msgr2=0x7f0dec0b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/38519923 shutdown_connections 2024-04-02T16:11:45.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/38519923 wait complete. 2024-04-02T16:11:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 Processor -- start 2024-04-02T16:11:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- start start 2024-04-02T16:11:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 0x7f0dec1e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec148550 0x7f0dec1e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec073450 con 0x7f0dec072ec0 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dec09fee0 con 0x7f0dec148550 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0dec154e10 con 0x7f0dec09f0e0 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42630/0 (socket says 172.21.15.40:42630) 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 -- 172.21.15.40:0/2257684487 learned_addr learned my addr 172.21.15.40:0/2257684487 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de3fff640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 0x7f0dec1e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dead9d640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec148550 0x7f0dec1e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 -- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 msgr2=0x7f0dec1e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 0x7f0dec1e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 -- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec148550 msgr2=0x7f0dec1e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec148550 0x7f0dec1e0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 -- 172.21.15.40:0/2257684487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dec1e42d0 con 0x7f0dec072ec0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de3fff640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 0x7f0dec1e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0dea59c640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e960 tx=0x7f0ddc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc017070 con 0x7f0dec072ec0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00ca00 con 0x7f0dec072ec0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dec1e45b0 con 0x7f0dec072ec0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00cdb0 con 0x7f0dec072ec0 2024-04-02T16:11:45.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.132+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dec0b4160 con 0x7f0dec072ec0 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dec068d00 con 0x7f0dec072ec0 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0ddc01a070 con 0x7f0dec072ec0 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0de2ffd640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 0x7f0dcc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ddc087150 con 0x7f0dec072ec0 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0de3fff640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 0x7f0dcc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.136+0000 7f0de3fff640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 0x7f0dcc06a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dec13c730 tx=0x7f0dd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.140+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f0ddc0577a0 con 0x7f0dec072ec0 2024-04-02T16:11:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0dec072b90 con 0x7f0dec072ec0 2024-04-02T16:11:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0de2ffd640 1 -- 172.21.15.40:0/2257684487 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0ddc0570a0 con 0x7f0dec072ec0 2024-04-02T16:11:45.310 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 msgr2=0x7f0dcc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 0x7f0dcc06a870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dec13c730 tx=0x7f0dd4009040 comp rx=0 tx=0).stop 2024-04-02T16:11:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 msgr2=0x7f0dec13bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.304+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e960 tx=0x7f0ddc00ee20 comp rx=0 tx=0).stop 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 shutdown_connections 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec09f0e0 0x7f0dec1e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0dcc0683b0 0x7f0dcc06a870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec148550 0x7f0dec1e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 --2- 172.21.15.40:0/2257684487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dec072ec0 0x7f0dec13bf60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 >> 172.21.15.40:0/2257684487 conn(0x7f0dec0b7d10 msgr2=0x7f0dec0b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 shutdown_connections 2024-04-02T16:11:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.308+0000 7f0df0c09640 1 -- 172.21.15.40:0/2257684487 wait complete. 2024-04-02T16:11:45.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:45.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.440+0000 7fbf57615640 1 Processor -- start 2024-04-02T16:11:45.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 -- start start 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf50151830 0x7fbf50153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50154160 0x7fbf50156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf500af670 con 0x7fbf500ae320 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbf500af950 con 0x7fbf50151830 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf57615640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500aff10 con 0x7fbf50154160 2024-04-02T16:11:45.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf553b1640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf500ae320 0x7fbf5006c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60748/0 (socket says 172.21.15.40:60748) 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf553b1640 1 -- 172.21.15.40:0/3201920001 learned_addr learned my addr 172.21.15.40:0/3201920001 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 --2- 172.21.15.40:0/3201920001 >> v2:172.21.15.40:3300/0 conn(0x7fbf50151830 0x7fbf50153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf55bb2640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50154160 0x7fbf50156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf4ffff640 1 -- 172.21.15.40:0/3201920001 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197064680 0 0) 0x7fbf500af670 con 0x7fbf500ae320 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf4ffff640 1 -- 172.21.15.40:0/3201920001 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf38003530 con 0x7fbf500ae320 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 -- 172.21.15.40:0/3201920001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50154160 msgr2=0x7fbf50156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50154160 0x7fbf50156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 -- 172.21.15.40:0/3201920001 >> v1:172.21.15.40:6789/0 conn(0x7fbf500ae320 legacy=0x7fbf5006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 -- 172.21.15.40:0/3201920001 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500b5260 con 0x7fbf50151830 2024-04-02T16:11:45.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf54bb0640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50151830 0x7fbf50153c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff33fc975f6583c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf4ffff640 1 -- 172.21.15.40:0/3201920001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440b2070 con 0x7fbf50151830 2024-04-02T16:11:45.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf4ffff640 1 -- 172.21.15.40:0/3201920001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf440a5910 con 0x7fbf50151830 2024-04-02T16:11:45.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.444+0000 7fbf4ffff640 1 -- 172.21.15.40:0/3201920001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf440a5cc0 con 0x7fbf50151830 2024-04-02T16:11:45.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/3201920001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50151830 msgr2=0x7fbf50153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50151830 0x7fbf50153c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4401c8b0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/3201920001 shutdown_connections 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50154160 0x7fbf50156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- 172.21.15.40:0/3201920001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50151830 0x7fbf50153c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/3201920001 >> 172.21.15.40:0/3201920001 conn(0x7fbf500b8fd0 msgr2=0x7fbf500b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/3201920001 shutdown_connections 2024-04-02T16:11:45.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/3201920001 wait complete. 2024-04-02T16:11:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 Processor -- start 2024-04-02T16:11:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- start start 2024-04-02T16:11:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf500ae320 0x7fbf50149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50151830 0x7fbf501e4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500af2f0 con 0x7fbf50154160 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500b0300 con 0x7fbf500ae320 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500aeef0 con 0x7fbf50151830 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42640/0 (socket says 172.21.15.40:42640) 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 -- 172.21.15.40:0/1522756732 learned_addr learned my addr 172.21.15.40:0/1522756732 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf54bb0640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50151830 0x7fbf501e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 -- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50151830 msgr2=0x7fbf501e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50151830 0x7fbf501e4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 -- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf500ae320 msgr2=0x7fbf50149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf500ae320 0x7fbf50149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 -- 172.21.15.40:0/1522756732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf50156430 con 0x7fbf50154160 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf55bb2640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e880 tx=0x7fbf4000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017070 con 0x7fbf50154160 2024-04-02T16:11:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4000c940 con 0x7fbf50154160 2024-04-02T16:11:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf500b0900 con 0x7fbf50154160 2024-04-02T16:11:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000cd10 con 0x7fbf50154160 2024-04-02T16:11:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.448+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf50155140 con 0x7fbf50154160 2024-04-02T16:11:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf50155630 con 0x7fbf50154160 2024-04-02T16:11:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbf4001a070 con 0x7fbf50154160 2024-04-02T16:11:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf4dffb640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 0x7fbf3c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbf40086910 con 0x7fbf50154160 2024-04-02T16:11:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf553b1640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 0x7fbf3c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.452+0000 7fbf553b1640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 0x7fbf3c06a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf48007fd0 tx=0x7fbf48007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.456+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fbf40056f60 con 0x7fbf50154160 2024-04-02T16:11:45.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf500685f0 con 0x7fbf50154160 2024-04-02T16:11:45.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf4dffb640 1 -- 172.21.15.40:0/1522756732 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7fbf40056860 con 0x7fbf50154160 2024-04-02T16:11:45.657 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:45.657 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:11:45.657 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 msgr2=0x7fbf3c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 0x7fbf3c06a840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf48007fd0 tx=0x7fbf48007900 comp rx=0 tx=0).stop 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 msgr2=0x7fbf50147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e880 tx=0x7fbf4000ed40 comp rx=0 tx=0).stop 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 shutdown_connections 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf50151830 0x7fbf501e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbf3c068380 0x7fbf3c06a840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf50154160 0x7fbf50147c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 --2- 172.21.15.40:0/1522756732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf500ae320 0x7fbf50149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 >> 172.21.15.40:0/1522756732 conn(0x7fbf500b8fd0 msgr2=0x7fbf50153900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 shutdown_connections 2024-04-02T16:11:45.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.652+0000 7fbf57615640 1 -- 172.21.15.40:0/1522756732 wait complete. 2024-04-02T16:11:45.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 Processor -- start 2024-04-02T16:11:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 -- start start 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59480b23d0 0x7f59480b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 0x7f594809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f594809dfe0 0x7f5948148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f59480b19e0 con 0x7f59480b23d0 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f59480b05c0 con 0x7f594809dfe0 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594cc51640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59480b08a0 con 0x7f5948073230 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59480b23d0 0x7f59480b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59480b23d0 0x7f59480b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42656/0 (socket says 172.21.15.40:42656) 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 -- 172.21.15.40:0/296532289 learned_addr learned my addr 172.21.15.40:0/296532289 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f5946d9d640 1 --2- 172.21.15.40:0/296532289 >> v2:172.21.15.204:3300/0 conn(0x7f594809dfe0 0x7f5948148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f5945d9b640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 0x7f594809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 -- 172.21.15.40:0/296532289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 msgr2=0x7f594809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 0x7f594809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 -- 172.21.15.40:0/296532289 >> v2:172.21.15.204:3300/0 conn(0x7f594809dfe0 msgr2=0x7f5948148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 --2- 172.21.15.40:0/296532289 >> v2:172.21.15.204:3300/0 conn(0x7f594809dfe0 0x7f5948148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 -- 172.21.15.40:0/296532289 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59480b0b80 con 0x7f59480b23d0 2024-04-02T16:11:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f5945d9b640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 0x7f594809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594659c640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59480b23d0 0x7f59480b27b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f594806be30 tx=0x7f594001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47af1b1ed37fc271 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594559a640 1 -- 172.21.15.40:0/296532289 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400a56c0 con 0x7f59480b23d0 2024-04-02T16:11:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594559a640 1 -- 172.21.15.40:0/296532289 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59400a5860 con 0x7f59480b23d0 2024-04-02T16:11:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.780+0000 7f594559a640 1 -- 172.21.15.40:0/296532289 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400a5c30 con 0x7f59480b23d0 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/296532289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59480b23d0 msgr2=0x7f59480b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59480b23d0 0x7f59480b27b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f594806be30 tx=0x7f594001c960 comp rx=0 tx=0).stop 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/296532289 shutdown_connections 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- 172.21.15.40:0/296532289 >> v2:172.21.15.204:3300/0 conn(0x7f594809dfe0 0x7f5948148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5948073230 0x7f594809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- 172.21.15.40:0/296532289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59480b23d0 0x7f59480b27b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/296532289 >> 172.21.15.40:0/296532289 conn(0x7f59480b8fd0 msgr2=0x7f59480b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/296532289 shutdown_connections 2024-04-02T16:11:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/296532289 wait complete. 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 Processor -- start 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- start start 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 0x7f594813a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59480b23d0 0x7f594813a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59481460f0 con 0x7f5948073230 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5948145f70 con 0x7f594809dfe0 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5948146270 con 0x7f59480b23d0 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42658/0 (socket says 172.21.15.40:42658) 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 -- 172.21.15.40:0/413607922 learned_addr learned my addr 172.21.15.40:0/413607922 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f5945d9b640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 0x7f594813a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f5946d9d640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59480b23d0 0x7f594813a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 -- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59480b23d0 msgr2=0x7f594813a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59480b23d0 0x7f594813a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 -- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 msgr2=0x7f594813a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 0x7f594813a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 -- 172.21.15.40:0/413607922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594809b2f0 con 0x7f5948073230 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f5945d9b640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 0x7f594813a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594659c640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f594001b010 tx=0x7f5948146560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400bc030 con 0x7f5948073230 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59400b8470 con 0x7f5948073230 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59480ae960 con 0x7f5948073230 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400b8860 con 0x7f5948073230 2024-04-02T16:11:45.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594814f010 con 0x7f5948073230 2024-04-02T16:11:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.784+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59480685f0 con 0x7f5948073230 2024-04-02T16:11:45.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.788+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59400bf070 con 0x7f5948073230 2024-04-02T16:11:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.788+0000 7f593effd640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 0x7f592c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.788+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59401214f0 con 0x7f5948073230 2024-04-02T16:11:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.788+0000 7f5945d9b640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 0x7f592c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:45.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.788+0000 7f5945d9b640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 0x7f592c06a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59481daf00 tx=0x7f5930008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:45.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.792+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f59400c6070 con 0x7f5948073230 2024-04-02T16:11:45.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.952+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59480b1690 con 0x7f5948073230 2024-04-02T16:11:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f593effd640 1 -- 172.21.15.40:0/413607922 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f59400165e0 con 0x7f5948073230 2024-04-02T16:11:45.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:45.960 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 msgr2=0x7f592c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 0x7f592c06a840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59481daf00 tx=0x7f5930008040 comp rx=0 tx=0).stop 2024-04-02T16:11:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 msgr2=0x7f5948148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:45.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f594001b010 tx=0x7f5948146560 comp rx=0 tx=0).stop 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 shutdown_connections 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59480b23d0 0x7f594813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f592c068380 0x7f592c06a840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f594809dfe0 0x7f594813a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 --2- 172.21.15.40:0/413607922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5948073230 0x7f5948148080 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 >> 172.21.15.40:0/413607922 conn(0x7f59480b8fd0 msgr2=0x7f59481427d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 shutdown_connections 2024-04-02T16:11:45.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:45.956+0000 7f594cc51640 1 -- 172.21.15.40:0/413607922 wait complete. 2024-04-02T16:11:45.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:11:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 Processor -- start 2024-04-02T16:11:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 -- start start 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3818141520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3818141a60 0x7f381814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381814bac0 con 0x7f381814b630 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f38180b6120 con 0x7f3818141a60 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381f74e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3818141e80 con 0x7f38180bd9a0 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381dceb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f381814b630 0x7f38180b79b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60750/0 (socket says 172.21.15.40:60750) 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381dceb640 1 -- 172.21.15.40:0/2791022266 learned_addr learned my addr 172.21.15.40:0/2791022266 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3818141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381cce9640 1 --2- 172.21.15.40:0/2791022266 >> v2:172.21.15.204:3300/0 conn(0x7f3818141a60 0x7f381814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051158256 0 0) 0x7f381814bac0 con 0x7f381814b630 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3804003530 con 0x7f381814b630 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1672966582 0 0) 0x7f3804003530 con 0x7f381814b630 2024-04-02T16:11:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 msgr2=0x7f3818141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3818141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 >> v2:172.21.15.204:3300/0 conn(0x7f3818141a60 msgr2=0x7f381814b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 --2- 172.21.15.40:0/2791022266 >> v2:172.21.15.204:3300/0 conn(0x7f3818141a60 0x7f381814b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180bdd80 con 0x7f381814b630 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f3810001a00 con 0x7f381814b630 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 >> v1:172.21.15.40:6789/0 conn(0x7f381814b630 legacy=0x7f38180b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38040039c0 0x7f3804003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3804004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3804004d50 0x7f38040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38180b6120 con 0x7f38040039c0 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3818141e80 con 0x7f3804004d50 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f380400ba50 con 0x7f38180bd9a0 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38040039c0 0x7f3804003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381cce9640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3804004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381dceb640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3804004d50 0x7f38040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 -- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 msgr2=0x7f3804004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38180bd9a0 0x7f3804004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 -- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3804004d50 msgr2=0x7f38040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3804004d50 0x7f38040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 -- 172.21.15.40:0/2791022266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3804003530 con 0x7f38040039c0 2024-04-02T16:11:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381dceb640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3804004d50 0x7f38040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f381d4ea640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38040039c0 0x7f3804003de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3814009ac0 tx=0x7f381401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff7e50bd7e1322a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814009690 con 0x7f38040039c0 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38140a6430 con 0x7f38040039c0 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.088+0000 7f380ffff640 1 -- 172.21.15.40:0/2791022266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38140a6800 con 0x7f38040039c0 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/2791022266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38040039c0 msgr2=0x7f3804003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 --2- 172.21.15.40:0/2791022266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38040039c0 0x7f3804003de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3814009ac0 tx=0x7f381401ca50 comp rx=0 tx=0).stop 2024-04-02T16:11:46.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 -- 172.21.15.40:0/2791022266 reap_dead start 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/2791022266 shutdown_connections 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/2791022266 >> 172.21.15.40:0/2791022266 conn(0x7f3818073310 msgr2=0x7f38180bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/2791022266 shutdown_connections 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/2791022266 wait complete. 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 Processor -- start 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- start start 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38040039c0 0x7f38181d3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38181e34e0 0x7f38181e3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38180b4810 con 0x7f38181d7210 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38180b9840 con 0x7f38181e34e0 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f38180b4990 con 0x7f38040039c0 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42676/0 (socket says 172.21.15.40:42676) 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 -- 172.21.15.40:0/448655648 learned_addr learned my addr 172.21.15.40:0/448655648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381d4ea640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38040039c0 0x7f38181d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381cce9640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38181e34e0 0x7f38181e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 -- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38040039c0 msgr2=0x7f38181d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38040039c0 0x7f38181d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 -- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38181e34e0 msgr2=0x7f38181e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38181e34e0 0x7f38181e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 -- 172.21.15.40:0/448655648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38181e3ea0 con 0x7f38181d7210 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381dceb640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f38180b4320 tx=0x7f381000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810004d70 con 0x7f38181d7210 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381000bb80 con 0x7f38181d7210 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38181d4bd0 con 0x7f38181d7210 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810007110 con 0x7f38181d7210 2024-04-02T16:11:46.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3818144ec0 con 0x7f38181d7210 2024-04-02T16:11:46.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.092+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38180685f0 con 0x7f38181d7210 2024-04-02T16:11:46.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.096+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f381001b020 con 0x7f38181d7210 2024-04-02T16:11:46.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.096+0000 7f380dffb640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 0x7f37fc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.096+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3810086b60 con 0x7f38181d7210 2024-04-02T16:11:46.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.096+0000 7f381d4ea640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 0x7f37fc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.096+0000 7f381d4ea640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 0x7f37fc06a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38140ad000 tx=0x7f38140b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.104+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f38181e3ea0 con 0x7f38181d7210 2024-04-02T16:11:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.288+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38181d7bd0 con 0x7f38181d7210 2024-04-02T16:11:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.288+0000 7f380dffb640 1 -- 172.21.15.40:0/448655648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7f3810057110 con 0x7f38181d7210 2024-04-02T16:11:46.294 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:46.294 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:11:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:11:46.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 msgr2=0x7f37fc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 0x7f37fc06a870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38140ad000 tx=0x7f38140b2040 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 msgr2=0x7f38181d7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f38180b4320 tx=0x7f381000b4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 shutdown_connections 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37fc0683b0 0x7f37fc06a870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38181d7210 0x7f38181d7690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38181e34e0 0x7f38181e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 --2- 172.21.15.40:0/448655648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38040039c0 0x7f38181d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 >> 172.21.15.40:0/448655648 conn(0x7f3818073310 msgr2=0x7f3818147f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 shutdown_connections 2024-04-02T16:11:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.292+0000 7f381f74e640 1 -- 172.21.15.40:0/448655648 wait complete. 2024-04-02T16:11:46.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T16:11:31.473429+0000', 'modified': '2024-04-02T16:11:34.552731+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4229': {'gid': 4229, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/905036532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 905036532}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 905036532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4229, 'qdb_cluster': [4229]} 2024-04-02T16:11:46.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T16:11:46.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T16:11:46.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 Processor -- start 2024-04-02T16:11:46.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 -- start start 2024-04-02T16:11:46.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34a00b3500 0x7f34a00b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a00b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a00b6310 con 0x7f34a00b4bf0 2024-04-02T16:11:46.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f34a00b55d0 con 0x7f34a00b3500 2024-04-02T16:11:46.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f34a5861640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34a014b560 con 0x7f34a00b8c80 2024-04-02T16:11:46.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a00b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349e7fc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34a00b3500 0x7f34a00b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349effd640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f34a00b4bf0 0x7f34a014ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:49940/0 (socket says 172.21.15.40:49940) 2024-04-02T16:11:46.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349effd640 1 -- 172.21.15.40:0/49377020 learned_addr learned my addr 172.21.15.40:0/49377020 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:46.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349e7fc640 1 -- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 msgr2=0x7f34a00b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349e7fc640 1 --2- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a00b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349e7fc640 1 -- 172.21.15.40:0/49377020 >> v1:172.21.15.204:6789/0 conn(0x7f34a00b4bf0 legacy=0x7f34a014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.640+0000 7f349e7fc640 1 -- 172.21.15.40:0/49377020 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a00b3fa0 con 0x7f34a00b3500 2024-04-02T16:11:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f349e7fc640 1 --2- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b3500 0x7f34a00b4fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34940097c0 tx=0x7f349401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6194b1cd4e6e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f349dffb640 1 -- 172.21.15.40:0/49377020 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940b0070 con 0x7f34a00b3500 2024-04-02T16:11:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f349dffb640 1 -- 172.21.15.40:0/49377020 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34940ab600 con 0x7f34a00b3500 2024-04-02T16:11:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f349dffb640 1 -- 172.21.15.40:0/49377020 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940ab9d0 con 0x7f34a00b3500 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b3500 msgr2=0x7f34a00b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b3500 0x7f34a00b4fd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34940097c0 tx=0x7f349401c380 comp rx=0 tx=0).stop 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- 172.21.15.40:0/49377020 shutdown_connections 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a00b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- 172.21.15.40:0/49377020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b3500 0x7f34a00b4fd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- 172.21.15.40:0/49377020 >> 172.21.15.40:0/49377020 conn(0x7f34a00734b0 msgr2=0x7f34a00bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- 172.21.15.40:0/49377020 shutdown_connections 2024-04-02T16:11:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- 172.21.15.40:0/49377020 wait complete. 2024-04-02T16:11:46.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 Processor -- start 2024-04-02T16:11:46.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- start start 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b4bf0 0x7f34a01471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a01e3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0154c10 con 0x7f34a00b3500 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0154d90 con 0x7f34a00b4bf0 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.644+0000 7f34a5861640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f34a0154f10 con 0x7f34a00b8c80 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42682/0 (socket says 172.21.15.40:42682) 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 -- 172.21.15.40:0/608299262 learned_addr learned my addr 172.21.15.40:0/608299262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349e7fc640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b4bf0 0x7f34a01471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349f7fe640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a01e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 -- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 msgr2=0x7f34a01e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a01e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 -- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b4bf0 msgr2=0x7f34a01471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b4bf0 0x7f34a01471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 -- 172.21.15.40:0/608299262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a0142d50 con 0x7f34a00b3500 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349effd640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f349000cb10 tx=0x7f349000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 -- 172.21.15.40:0/608299262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490017070 con 0x7f34a00b3500 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 -- 172.21.15.40:0/608299262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349000ac70 con 0x7f34a00b3500 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a0143310 con 0x7f34a00b3500 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 -- 172.21.15.40:0/608299262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490012620 con 0x7f34a00b3500 2024-04-02T16:11:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01e0460 con 0x7f34a00b3500 2024-04-02T16:11:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 -- 172.21.15.40:0/608299262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f349001a070 con 0x7f34a00b3500 2024-04-02T16:11:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 0x7f348c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f3487fff640 1 -- 172.21.15.40:0/608299262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34900866f0 con 0x7f34a00b3500 2024-04-02T16:11:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f349e7fc640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 0x7f348c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.648+0000 7f34a89e0000 1 Processor -- start 2024-04-02T16:11:46.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- start start 2024-04-02T16:11:46.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349e7fc640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 0x7f348c06a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f34940ad000 tx=0x7f3494008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f132760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f13d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56283ef4f980 con 0x56283f107cd0 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56283eeecbd0 con 0x56283f132380 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56283ef4b930 con 0x56283f132ca0 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42684/0 (socket says 172.21.15.40:42684) 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 -- 172.21.15.40:0/3031989103 learned_addr learned my addr 172.21.15.40:0/3031989103 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349f7fe640 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349e7fc640 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f132760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 -- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 msgr2=0x56283f13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 -- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 msgr2=0x56283f132760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f132760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 -- 172.21.15.40:0/3031989103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283ef29c30 con 0x56283f107cd0 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3490084c50 tx=0x7f3490001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54e5c89620497f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34857fa640 1 -- 172.21.15.40:0/3031989103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349005f1a0 con 0x56283f107cd0 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34857fa640 1 -- 172.21.15.40:0/3031989103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900548c0 con 0x56283f107cd0 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34857fa640 1 -- 172.21.15.40:0/3031989103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490052c30 con 0x56283f107cd0 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031989103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 msgr2=0x56283f130850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3490084c50 tx=0x7f3490001260 comp rx=0 tx=0).stop 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031989103 shutdown_connections 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f132760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031989103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f130850 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031989103 >> 172.21.15.40:0/3031989103 conn(0x56283ec13780 msgr2=0x56283f130140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031989103 shutdown_connections 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031989103 wait complete. 2024-04-02T16:11:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- start start 2024-04-02T16:11:46.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 Processor -- start 2024-04-02T16:11:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 0x56283f103060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56283ef0d530 con 0x56283f132ca0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56283ef2a7f0 con 0x56283f132380 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56283ef5c090 con 0x56283f107cd0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f34a89e0000 10 client.0 _ll_register_callbacks cb 0x56283efb1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42690/0 (socket says 172.21.15.40:42690) 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349f7fe640 1 -- 172.21.15.40:0/4288269135 learned_addr learned my addr 172.21.15.40:0/4288269135 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349effd640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 0x56283f103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.652+0000 7f349e7fc640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 -- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 msgr2=0x56283f103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 0x56283f103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 -- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 msgr2=0x56283f1035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f1035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 -- 172.21.15.40:0/4288269135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283ef95750 con 0x56283f132ca0 2024-04-02T16:11:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349effd640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 0x56283f103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349f7fe640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3498009ca0 tx=0x7f34980136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33179fe85ba8ea6a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349801a030 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3498016460 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34900548c0 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f34980059a0 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 10 client.4638 fetch_fsmap learned FSMap version 7 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x56283eee77b0 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900548c0 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) ==== 2284+0+0 (secure 0 0 0) 0x7f3498029080 con 0x56283f132ca0 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 10 client.4638 fetch_fsmap finished waiting for FSMap version 7 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 20 client.4638 populate_metadata read hostname 'smithi040' 2024-04-02T16:11:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f145920 0x56283f147d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 4 client.4638 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 --> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283eeeabc0 con 0x56283f145920 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349effd640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f145920 0x56283f147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f349effd640 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f145920 0x56283f147d10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd83e4a9465c45 server_cookie=f86e5288442385ec in_seq=0 out_seq=0 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 10 client.4638 ms_handle_connect on v2:172.21.15.204:6834/905036532 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 1 -- 172.21.15.40:0/4288269135 <== mds.0 v2:172.21.15.204:6834/905036532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56283eeeabc0 con 0x56283f145920 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f347e7fc640 10 client.4638 handle_command_reply: tid=0 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 client.4638 shutdown 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 msgr2=0x56283f103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.656+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3498009ca0 tx=0x7f34980136b0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 shutdown_connections 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f145920 0x56283f147d10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132ca0 0x56283f103ae0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132380 0x56283f1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 --2- 172.21.15.40:0/4288269135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f107cd0 0x56283f103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 >> 172.21.15.40:0/4288269135 conn(0x56283ec13780 msgr2=0x56283f130140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f347e7fc640 0 client.4638 ms_handle_reset on v2:172.21.15.204:6834/905036532 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 shutdown_connections 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 1 -- 172.21.15.40:0/4288269135 wait complete. 2024-04-02T16:11:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.660+0000 7f34a89e0000 20 client.4638 trim_cache size 0 max 16384 2024-04-02T16:11:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 Processor -- start 2024-04-02T16:11:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- start start 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132380 0x56283f14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132ca0 0x56283f131100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01ec7b0 con 0x56283f107cd0 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34a00bc590 con 0x56283f132ca0 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56283ef4ec90 con 0x56283f132380 2024-04-02T16:11:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42698/0 (socket says 172.21.15.40:42698) 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 -- 172.21.15.40:0/689355754 learned_addr learned my addr 172.21.15.40:0/689355754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349f7fe640 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132ca0 0x56283f131100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 -- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132380 msgr2=0x56283f14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132380 0x56283f14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 -- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132ca0 msgr2=0x56283f131100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132ca0 0x56283f131100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 -- 172.21.15.40:0/689355754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283eed7e80 con 0x56283f107cd0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f349000cae0 tx=0x7f3490052990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8830693024bc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f3484ff9640 1 -- 172.21.15.40:0/689355754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490079020 con 0x56283f107cd0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f3484ff9640 1 -- 172.21.15.40:0/689355754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900750c0 con 0x56283f107cd0 2024-04-02T16:11:46.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f3484ff9640 1 -- 172.21.15.40:0/689355754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490084070 con 0x56283f107cd0 2024-04-02T16:11:46.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/689355754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 msgr2=0x56283f14a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f349000cae0 tx=0x7f3490052990 comp rx=0 tx=0).stop 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/689355754 shutdown_connections 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132380 0x56283f14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f132ca0 0x56283f131100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- 172.21.15.40:0/689355754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f107cd0 0x56283f14a960 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/689355754 >> 172.21.15.40:0/689355754 conn(0x56283ec13780 msgr2=0x56283f12d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/689355754 shutdown_connections 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/689355754 wait complete. 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- start start 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 Processor -- start 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 0x56283f149430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x56283ef062a0 con 0x56283f132380 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56283ef53fb0 con 0x56283f107cd0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x56283ef06440 con 0x56283f132ca0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 10 client.0 _ll_register_callbacks cb 0x7f3498009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42710/0 (socket says 172.21.15.40:42710) 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 -- 172.21.15.40:0/3031670415 learned_addr learned my addr 172.21.15.40:0/3031670415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 0x56283f149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349f7fe640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 -- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 msgr2=0x56283f149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 -- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 msgr2=0x56283f149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 0x56283f149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 -- 172.21.15.40:0/3031670415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283ef021f0 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 0x56283f149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349e7fc640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f349401c860 tx=0x7f349401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214a78dabb61b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34940b0040 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f34940b6d50 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34900750c0 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f34940d1050 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 10 client.4644 fetch_fsmap learned FSMap version 7 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x56283eef62a0 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900750c0 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) ==== 2284+0+0 (secure 0 0 0) 0x7f34940d4020 con 0x56283f132380 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 10 client.4644 fetch_fsmap finished waiting for FSMap version 7 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 10 client.4644 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 20 client.4644 populate_metadata read hostname 'smithi040' 2024-04-02T16:11:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f1abc60 0x56283f1ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 4 client.4644 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 --> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] -- command(tid 0: {"prefix": "status"}) -- 0x56283ef5cec0 con 0x56283f1abc60 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f1abc60 0x56283f1ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f349effd640 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f1abc60 0x56283f1ae050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d71908e9fd2359 server_cookie=b4349d7803ccb2b9 in_seq=0 out_seq=0 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.680+0000 7f347e7fc640 10 client.4644 ms_handle_connect on v2:172.21.15.204:6834/905036532 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f347e7fc640 1 -- 172.21.15.40:0/3031670415 <== mds.0 v2:172.21.15.204:6834/905036532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x56283ef5cec0 con 0x56283f1abc60 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f347e7fc640 10 client.4644 handle_command_reply: tid=0 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 client.4644 shutdown 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 msgr2=0x56283f149970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f349401c860 tx=0x7f349401abc0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 shutdown_connections 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:6834/905036532,v1:172.21.15.204:6835/905036532] conn(0x56283f1abc60 0x56283f1ae050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x56283f132ca0 0x56283f149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56283f132380 0x56283f149970 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 --2- 172.21.15.40:0/3031670415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56283f107cd0 0x56283f149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f347e7fc640 0 client.4644 ms_handle_reset on v2:172.21.15.204:6834/905036532 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 >> 172.21.15.40:0/3031670415 conn(0x56283ec13780 msgr2=0x56283f131100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 shutdown_connections 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 1 -- 172.21.15.40:0/3031670415 wait complete. 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a89e0000 20 client.4644 trim_cache size 0 max 16384 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "f7c926e5-333e-4244-9af2-414c6ebf5077", 2024-04-02T16:11:46.693 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4229, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 14.537458853, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 7, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 21.378884605 2024-04-02T16:11:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 msgr2=0x7f348c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 0x7f348c06a870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f34940ad000 tx=0x7f3494008040 comp rx=0 tx=0).stop 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 msgr2=0x7f34a01e0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f349000cb10 tx=0x7f349000a060 comp rx=0 tx=0).stop 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 shutdown_connections 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a00b8c80 0x7f34a01e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f348c0683b0 0x7f348c06a870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a00b4bf0 0x7f34a01471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 --2- 172.21.15.40:0/608299262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f34a00b3500 0x7f34a01e0c60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 >> 172.21.15.40:0/608299262 conn(0x7f34a00734b0 msgr2=0x7f34a00ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 shutdown_connections 2024-04-02T16:11:46.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.684+0000 7f34a5861640 1 -- 172.21.15.40:0/608299262 wait complete. 2024-04-02T16:11:46.705 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T16:11:46.714 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T16:11:46.715 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T16:11:46.715 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T16:11:46.715 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-02T16:11:46.738 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-02T16:11:46.739 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-02T16:11:46.739 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-02T16:11:46.739 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T16:11:46.739 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:46.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:46.739 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-02T16:11:46.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:46.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-02T16:11:46.801 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-02T16:11:46.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:11:46.803 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-02T16:11:46.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:11:46.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 Processor -- start 2024-04-02T16:11:46.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- start start 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26d006c8a0 0x7f26d00af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d00ae830 con 0x7f26d00b3800 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d006cc80 con 0x7f26d00b43a0 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f26d00b2b50 con 0x7f26d006c8a0 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d4fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26d00b3800 0x7f26d00b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60754/0 (socket says 172.21.15.40:60754) 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d4fff640 1 -- 172.21.15.40:0/3306802440 learned_addr learned my addr 172.21.15.40:0/3306802440 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:46.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d5800640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26d006c8a0 0x7f26d00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d5800640 1 -- 172.21.15.40:0/3306802440 >> v1:172.21.15.204:6789/0 conn(0x7f26d00b43a0 legacy=0x7f26d014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:46.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d5800640 1 -- 172.21.15.40:0/3306802440 >> v1:172.21.15.40:6789/0 conn(0x7f26d00b3800 legacy=0x7f26d00b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d5800640 1 -- 172.21.15.40:0/3306802440 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d00adfa0 con 0x7f26d006c8a0 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d5800640 1 --2- 172.21.15.40:0/3306802440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d006c8a0 0x7f26d00af150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097c0 tx=0x7f26c001e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6519351d20692a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26c7fff640 1 -- 172.21.15.40:0/3306802440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00b2070 con 0x7f26d006c8a0 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26c7fff640 1 -- 172.21.15.40:0/3306802440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c00ad600 con 0x7f26d006c8a0 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26c7fff640 1 -- 172.21.15.40:0/3306802440 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00ad9d0 con 0x7f26d006c8a0 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- 172.21.15.40:0/3306802440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d006c8a0 msgr2=0x7f26d00af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 --2- 172.21.15.40:0/3306802440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d006c8a0 0x7f26d00af150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26c00097c0 tx=0x7f26c001e5e0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- 172.21.15.40:0/3306802440 shutdown_connections 2024-04-02T16:11:46.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 --2- 172.21.15.40:0/3306802440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d006c8a0 0x7f26d00af150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- 172.21.15.40:0/3306802440 >> 172.21.15.40:0/3306802440 conn(0x7f26d00b8fd0 msgr2=0x7f26d00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:46.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- 172.21.15.40:0/3306802440 shutdown_connections 2024-04-02T16:11:46.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- 172.21.15.40:0/3306802440 wait complete. 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 Processor -- start 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.960+0000 7f26d7a64640 1 -- start start 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26d006c8a0 0x7f26d01dc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d00b3800 0x7f26d009d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26d00b3130 con 0x7f26d00b43a0 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f26d00b54d0 con 0x7f26d00b3800 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f26d009acd0 con 0x7f26d006c8a0 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42726/0 (socket says 172.21.15.40:42726) 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 -- 172.21.15.40:0/2508001363 learned_addr learned my addr 172.21.15.40:0/2508001363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d4fff640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d00b3800 0x7f26d009d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d5800640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26d006c8a0 0x7f26d01dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 -- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26d006c8a0 msgr2=0x7f26d01dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26d006c8a0 0x7f26d01dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 -- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d00b3800 msgr2=0x7f26d009d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:46.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d00b3800 0x7f26d009d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 -- 172.21.15.40:0/2508001363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d01dc850 con 0x7f26d00b43a0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d6001640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b410 tx=0x7f26c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8015020 con 0x7f26d00b43a0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c8011870 con 0x7f26d00b43a0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d009cbd0 con 0x7f26d00b43a0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8011c60 con 0x7f26d00b43a0 2024-04-02T16:11:46.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d009c8f0 con 0x7f26d00b43a0 2024-04-02T16:11:46.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d01db920 con 0x7f26d00b43a0 2024-04-02T16:11:46.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f26c8018020 con 0x7f26d00b43a0 2024-04-02T16:11:46.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 0x7f26b406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:46.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26c8086460 con 0x7f26d00b43a0 2024-04-02T16:11:46.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d5800640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 0x7f26b406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:46.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26d5800640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 0x7f26b406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26c00af000 tx=0x7f26c00a8000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:46.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:46.968+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f26c801f030 con 0x7f26d00b43a0 2024-04-02T16:11:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.172+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f26d00685f0 con 0x7f26d00b43a0 2024-04-02T16:11:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.176+0000 7f26c5ffb640 1 -- 172.21.15.40:0/2508001363 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f26c8056ab0 con 0x7f26d00b43a0 2024-04-02T16:11:47.180 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 msgr2=0x7f26b406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 0x7f26b406a840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26c00af000 tx=0x7f26c00a8000 comp rx=0 tx=0).stop 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 msgr2=0x7f26d009d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b410 tx=0x7f26c800b8d0 comp rx=0 tx=0).stop 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 shutdown_connections 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f26b4068380 0x7f26b406a840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26d00b43a0 0x7f26d009d5a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d00b3800 0x7f26d009d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:47.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 --2- 172.21.15.40:0/2508001363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f26d006c8a0 0x7f26d01dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:47.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 >> 172.21.15.40:0/2508001363 conn(0x7f26d00b8fd0 msgr2=0x7f26d00b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:47.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 shutdown_connections 2024-04-02T16:11:47.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47.180+0000 7f26d7a64640 1 -- 172.21.15.40:0/2508001363 wait complete. 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:11:47.201 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:11:47.202 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:11:47.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:47.202 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-02T16:11:47.208 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:e4:3d:72:cc brd ff:ff:ff:ff:ff:ff 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-02T16:11:47.209 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:11:47.210 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:11:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:11:47.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:47.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:47.335 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:11:47.346 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-02T16:11:47.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:47.347 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-02T16:11:47.407 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:11:47.407 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:11:47.408 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:11:47.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:11:47.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:11:47.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:11:47.408 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:11:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:11:47.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:47.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:47.523 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-02T16:11:47.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:47.573 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-02T16:11:47.621 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:11:47.621 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:11:47.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:47.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T16:11:47.621 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:11:47.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:11:47.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:47.749 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T16:11:47.750 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:11:47.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:11:47.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:47.861 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:11:47.861 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:11:47.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-02T16:11:47.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T16:11:47.861 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:11:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:11:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:47.967 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-02T16:11:47.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:11:47.967 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:48.016 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:11:48.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:11:48.017 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:48.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:11:48.122 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:11:48.175 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:11:48.176 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:11:48.177 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:11:48.177 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:11:48.177 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:11:48.177 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:48.177 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:48.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:48.178 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:11:48.225 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T16:11:48.225 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-02T16:11:48.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:11:48.284 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:11:48.332 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:11:48.333 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:11:48.334 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:48.335 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:48.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:48.336 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:11:48.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2024-04-02T16:11:48.352+0000 7f430ff38580 -1 init, newargv = 0x55d963c057e0 newargc=15 2024-04-02T16:11:48.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[34695]: starting ceph client 2024-04-02T16:11:48.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[34695]: starting fuse 2024-04-02T16:11:49.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:11:49.473 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:11:49.526 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:11:49.526 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:11:49.526 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:11:49.526 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:11:49.527 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:11:49.527 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:11:49.527 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:11:49.527 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:11:49.527 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:11:49.529 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:11:49.530 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:11:49.531 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:49.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:49.534 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:11:49.581 INFO:teuthology.orchestra.run.smithi040.stdout:45 2024-04-02T16:11:49.581 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:11:49.581 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T16:11:49.582 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.0")) 2024-04-02T16:11:49.583 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-02T16:11:49.708 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.34695.asok 2024-04-02T16:11:49.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:11:49.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34695.asok 2024-04-02T16:11:49.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:49.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34695.asok status 2024-04-02T16:11:49.880 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "34695", 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-04-02T16:11:49.881 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4659, 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4659 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2408432887 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:11:49.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2408432887 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4659 192.168.0.1:0/2408432887", 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/2408432887", 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 7, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 20, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-04-02T16:11:49.883 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-04-02T16:11:49.884 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:11:49.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:49.895 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-04-02T16:11:49.895 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:49.901 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-04-02T16:11:49.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:49.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:11:49.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:11:49.959 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-02T16:11:49.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 Processor -- start 2024-04-02T16:11:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 -- start start 2024-04-02T16:11:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff068070d80 0x7ff0681477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff068148140 con 0x7ff0680b05c0 2024-04-02T16:11:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff068072760 con 0x7ff068147d30 2024-04-02T16:11:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff06dcba640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff068072a40 con 0x7ff068070d80 2024-04-02T16:11:50.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff066ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff068070d80 0x7ff0681477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff0677fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff0680b05c0 0x7ff06809ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60756/0 (socket says 172.21.15.40:60756) 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.124+0000 7ff0677fe640 1 -- 172.21.15.40:0/3491490658 learned_addr learned my addr 172.21.15.40:0/3491490658 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff066ffd640 1 -- 172.21.15.40:0/3491490658 >> v1:172.21.15.204:6789/0 conn(0x7ff068147d30 legacy=0x7ff068152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff066ffd640 1 -- 172.21.15.40:0/3491490658 >> v1:172.21.15.40:6789/0 conn(0x7ff0680b05c0 legacy=0x7ff06809ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff066ffd640 1 -- 172.21.15.40:0/3491490658 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068072d20 con 0x7ff068070d80 2024-04-02T16:11:50.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff0667fc640 1 -- 172.21.15.40:0/3491490658 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223133421 0 0) 0x7ff068148140 con 0x7ff0680b05c0 2024-04-02T16:11:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff066ffd640 1 --2- 172.21.15.40:0/3491490658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff068070d80 0x7ff0681477f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0580097c0 tx=0x7ff05801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145bfede7603de5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff0667fc640 1 -- 172.21.15.40:0/3491490658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580b0070 con 0x7ff068070d80 2024-04-02T16:11:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff0667fc640 1 -- 172.21.15.40:0/3491490658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580ab600 con 0x7ff068070d80 2024-04-02T16:11:50.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff0667fc640 1 -- 172.21.15.40:0/3491490658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a4e60 con 0x7ff068070d80 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 -- 172.21.15.40:0/3491490658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff068070d80 msgr2=0x7ff0681477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 --2- 172.21.15.40:0/3491490658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff068070d80 0x7ff0681477f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff0580097c0 tx=0x7ff05801c3d0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 -- 172.21.15.40:0/3491490658 shutdown_connections 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 --2- 172.21.15.40:0/3491490658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff068070d80 0x7ff0681477f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 -- 172.21.15.40:0/3491490658 >> 172.21.15.40:0/3491490658 conn(0x7ff0680b7d10 msgr2=0x7ff0680b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 -- 172.21.15.40:0/3491490658 shutdown_connections 2024-04-02T16:11:50.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.128+0000 7ff06dcba640 1 -- 172.21.15.40:0/3491490658 wait complete. 2024-04-02T16:11:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 Processor -- start 2024-04-02T16:11:50.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- start start 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 0x7ff0680b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff068147d30 0x7ff068148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680b0b20 con 0x7ff0680b05c0 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680b09a0 con 0x7ff068070d80 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff0681564c0 con 0x7ff068147d30 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42752/0 (socket says 172.21.15.40:42752) 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 -- 172.21.15.40:0/2817976585 learned_addr learned my addr 172.21.15.40:0/2817976585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff067fff640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff068147d30 0x7ff068148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff0677fe640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 0x7ff0680b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 -- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff068147d30 msgr2=0x7ff068148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff068147d30 0x7ff068148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 -- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 msgr2=0x7ff0680b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 0x7ff0680b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 -- 172.21.15.40:0/2817976585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06813a980 con 0x7ff0680b05c0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff0677fe640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 0x7ff0680b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff066ffd640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ae040 tx=0x7ff05801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580b0040 con 0x7ff0680b05c0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580a4e60 con 0x7ff0680b05c0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0680b2280 con 0x7ff0680b05c0 2024-04-02T16:11:50.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058004230 con 0x7ff0680b05c0 2024-04-02T16:11:50.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0680b1dc0 con 0x7ff0680b05c0 2024-04-02T16:11:50.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068068d00 con 0x7ff0680b05c0 2024-04-02T16:11:50.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff058003620 con 0x7ff0680b05c0 2024-04-02T16:11:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 0x7ff05406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.132+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff058121980 con 0x7ff0680b05c0 2024-04-02T16:11:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.136+0000 7ff0677fe640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 0x7ff05406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.136+0000 7ff0677fe640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 0x7ff05406a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f70 tx=0x7ff05c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.144+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff058016050 con 0x7ff0680b05c0 2024-04-02T16:11:50.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.304+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff068145b70 con 0x7ff0680b05c0 2024-04-02T16:11:50.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff04ffff640 1 -- 172.21.15.40:0/2817976585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7ff0580f2140 con 0x7ff0680b05c0 2024-04-02T16:11:50.309 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:50.309 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:50.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 msgr2=0x7ff05406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 0x7ff05406a840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff05c000f70 tx=0x7ff05c009040 comp rx=0 tx=0).stop 2024-04-02T16:11:50.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 msgr2=0x7ff06813a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ae040 tx=0x7ff05801ac10 comp rx=0 tx=0).stop 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 shutdown_connections 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff068147d30 0x7ff068148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff054068380 0x7ff05406a840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0680b05c0 0x7ff06813a1e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 --2- 172.21.15.40:0/2817976585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff068070d80 0x7ff0680b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 >> 172.21.15.40:0/2817976585 conn(0x7ff0680b7d10 msgr2=0x7ff0680b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 shutdown_connections 2024-04-02T16:11:50.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.308+0000 7ff06dcba640 1 -- 172.21.15.40:0/2817976585 wait complete. 2024-04-02T16:11:50.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:50.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:50.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 Processor -- start 2024-04-02T16:11:50.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 -- start start 2024-04-02T16:11:50.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfb00b8b60 0x7fdfb00b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdfb0147940 0x7fdfb0147d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb013e350 con 0x7fdfb00b52b0 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb01471d0 con 0x7fdfb00b8b60 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb64bf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb00b5b80 con 0x7fdfb0147940 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfb00b8b60 0x7fdfb00b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfb00b8b60 0x7fdfb00b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42766/0 (socket says 172.21.15.40:42766) 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 -- 172.21.15.40:0/1507078499 learned_addr learned my addr 172.21.15.40:0/1507078499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfb4a5c640 1 --2- 172.21.15.40:0/1507078499 >> v2:172.21.15.204:3300/0 conn(0x7fdfb0147940 0x7fdfb0147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 -- 172.21.15.40:0/1507078499 >> v2:172.21.15.204:3300/0 conn(0x7fdfb0147940 msgr2=0x7fdfb0147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 --2- 172.21.15.40:0/1507078499 >> v2:172.21.15.204:3300/0 conn(0x7fdfb0147940 0x7fdfb0147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 -- 172.21.15.40:0/1507078499 >> v1:172.21.15.204:6789/0 conn(0x7fdfb00b52b0 legacy=0x7fdfb013dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 -- 172.21.15.40:0/1507078499 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0147420 con 0x7fdfb00b8b60 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.444+0000 7fdfaffff640 1 --2- 172.21.15.40:0/1507078499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb00b8f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdfb006fc70 tx=0x7fdfa001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84885fb498675b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaeffd640 1 -- 172.21.15.40:0/1507078499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00b4070 con 0x7fdfb00b8b60 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaeffd640 1 -- 172.21.15.40:0/1507078499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa00a79b0 con 0x7fdfb00b8b60 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaeffd640 1 -- 172.21.15.40:0/1507078499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00a7d80 con 0x7fdfb00b8b60 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/1507078499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 msgr2=0x7fdfb00b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/1507078499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb00b8f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdfb006fc70 tx=0x7fdfa001eb60 comp rx=0 tx=0).stop 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/1507078499 shutdown_connections 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/1507078499 >> v2:172.21.15.204:3300/0 conn(0x7fdfb0147940 0x7fdfb0147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/1507078499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb00b8f40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/1507078499 >> 172.21.15.40:0/1507078499 conn(0x7fdfb00b5fb0 msgr2=0x7fdfb00b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/1507078499 shutdown_connections 2024-04-02T16:11:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/1507078499 wait complete. 2024-04-02T16:11:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 Processor -- start 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- start start 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb00b52b0 0x7fdfb0140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 0x7fdfb01406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00b7e90 con 0x7fdfb00b8b60 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00b7d10 con 0x7fdfb0147940 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00b8010 con 0x7fdfb00b52b0 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42770/0 (socket says 172.21.15.40:42770) 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 -- 172.21.15.40:0/641864431 learned_addr learned my addr 172.21.15.40:0/641864431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaffff640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb00b52b0 0x7fdfb0140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb4a5c640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 0x7fdfb01406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 -- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb00b52b0 msgr2=0x7fdfb0140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb00b52b0 0x7fdfb0140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 -- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 msgr2=0x7fdfb01406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 0x7fdfb01406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 -- 172.21.15.40:0/641864431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb00b1b80 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb4a5c640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 0x7fdfb01406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfaf7fe640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c460 tx=0x7fdfa400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4016020 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa40052d0 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb00b1e60 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4007260 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.448+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01401b0 con 0x7fdfb00b8b60 2024-04-02T16:11:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb0068d00 con 0x7fdfb00b8b60 2024-04-02T16:11:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdfa4007400 con 0x7fdfb00b8b60 2024-04-02T16:11:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfacff9640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 0x7fdf9c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfa40862c0 con 0x7fdfb00b8b60 2024-04-02T16:11:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfaffff640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 0x7fdf9c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.452+0000 7fdfaffff640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 0x7fdf9c06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00b1000 tx=0x7fdfa00a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.456+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fdfa4020020 con 0x7fdfb00b8b60 2024-04-02T16:11:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.628+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfb00b0660 con 0x7fdfb00b8b60 2024-04-02T16:11:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.628+0000 7fdfacff9640 1 -- 172.21.15.40:0/641864431 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdfa4005480 con 0x7fdfb00b8b60 2024-04-02T16:11:50.631 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 msgr2=0x7fdf9c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 0x7fdf9c06a840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00b1000 tx=0x7fdfa00a9070 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 msgr2=0x7fdfb01df9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400c460 tx=0x7fdfa400c920 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 shutdown_connections 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdf9c068380 0x7fdf9c06a840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfb0147940 0x7fdfb01406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfb00b8b60 0x7fdfb01df9d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 --2- 172.21.15.40:0/641864431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfb00b52b0 0x7fdfb0140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 >> 172.21.15.40:0/641864431 conn(0x7fdfb00b5fb0 msgr2=0x7fdfb00af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 shutdown_connections 2024-04-02T16:11:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.632+0000 7fdfb64bf640 1 -- 172.21.15.40:0/641864431 wait complete. 2024-04-02T16:11:50.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.768+0000 7fb8d37ef640 1 Processor -- start 2024-04-02T16:11:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.768+0000 7fb8d37ef640 1 -- start start 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d37ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8cc0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d37ef640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb8cc0b4370 0x7fb8cc14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d37ef640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8cc0ae830 con 0x7fb8cc0b3800 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d37ef640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb8cc0b2b50 con 0x7fb8cc0b4370 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d37ef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b0300 con 0x7fb8cc06c8a0 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8cc0b3800 0x7fb8cc0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60766/0 (socket says 172.21.15.40:60766) 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/3820103048 learned_addr learned my addr 172.21.15.40:0/3820103048 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d158b640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8cc0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d1d8c640 1 --2- 172.21.15.40:0/3820103048 >> v2:172.21.15.204:3300/0 conn(0x7fb8cc0b4370 0x7fb8cc14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307732022 0 0) 0x7fb8cc0ae830 con 0x7fb8cc0b3800 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b8003530 con 0x7fb8cc0b3800 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4010310651 0 0) 0x7fb8b8003530 con 0x7fb8cc0b3800 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 msgr2=0x7fb8cc0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8cc0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 >> v2:172.21.15.204:3300/0 conn(0x7fb8cc0b4370 msgr2=0x7fb8cc14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 --2- 172.21.15.40:0/3820103048 >> v2:172.21.15.204:3300/0 conn(0x7fb8cc0b4370 0x7fb8cc14f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc06cc80 con 0x7fb8cc0b3800 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fb8bc0016a0 con 0x7fb8cc0b3800 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 >> v1:172.21.15.40:6789/0 conn(0x7fb8cc0b3800 legacy=0x7fb8cc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8b80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b8003f40 0x7fb8b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8004d50 0x7fb8b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b2b50 con 0x7fb8b8003f40 2024-04-02T16:11:50.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b0300 con 0x7fb8b8004d50 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b800bab0 con 0x7fb8cc06c8a0 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b8003f40 0x7fb8b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d1d8c640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8004d50 0x7fb8b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d158b640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8b80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 msgr2=0x7fb8b80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8b80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8004d50 msgr2=0x7fb8b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8004d50 0x7fb8b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/3820103048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b8003530 con 0x7fb8b8003f40 2024-04-02T16:11:50.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d158b640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc06c8a0 0x7fb8b80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b8003f40 0x7fb8b8004810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c210 tx=0x7fb8bc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11a9201aa69355 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc007d70 con 0x7fb8b8003f40 2024-04-02T16:11:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc0a9020 con 0x7fb8b8003f40 2024-04-02T16:11:50.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.772+0000 7fb8c3fff640 1 -- 172.21.15.40:0/3820103048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc005ae0 con 0x7fb8b8003f40 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/3820103048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b8003f40 msgr2=0x7fb8b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/3820103048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8b8003f40 0x7fb8b8004810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c210 tx=0x7fb8bc01cca0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d1d8c640 1 -- 172.21.15.40:0/3820103048 reap_dead start 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/3820103048 shutdown_connections 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/3820103048 >> 172.21.15.40:0/3820103048 conn(0x7fb8cc0b8fd0 msgr2=0x7fb8cc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/3820103048 shutdown_connections 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/3820103048 wait complete. 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 Processor -- start 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- start start 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 0x7fb8cc1e1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc1e18d0 0x7fb8cc13f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b3130 con 0x7fb8cc1d3110 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b54d0 con 0x7fb8cc1e18d0 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc1400d0 con 0x7fb8cc142fe0 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42794/0 (socket says 172.21.15.40:42794) 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/2122769284 learned_addr learned my addr 172.21.15.40:0/2122769284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d158b640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 0x7fb8cc1e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 msgr2=0x7fb8cc1e1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 0x7fb8cc1e1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc1e18d0 msgr2=0x7fb8cc13f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc1e18d0 0x7fb8cc13f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 -- 172.21.15.40:0/2122769284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc09dd30 con 0x7fb8cc1d3110 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d158b640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 0x7fb8cc1e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d0d8a640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc01cce0 tx=0x7fb8b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc001cb0 con 0x7fb8cc1d3110 2024-04-02T16:11:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc0b2070 con 0x7fb8cc1d3110 2024-04-02T16:11:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc1e0e90 con 0x7fb8cc1d3110 2024-04-02T16:11:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc004d10 con 0x7fb8cc1d3110 2024-04-02T16:11:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc142160 con 0x7fb8cc1d3110 2024-04-02T16:11:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.776+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc0685f0 con 0x7fb8cc1d3110 2024-04-02T16:11:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.780+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8bc0a5870 con 0x7fb8cc1d3110 2024-04-02T16:11:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.780+0000 7fb8c1ffb640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 0x7fb8b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.780+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8bc11f6a0 con 0x7fb8cc1d3110 2024-04-02T16:11:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.780+0000 7fb8d158b640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 0x7fb8b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:50.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.780+0000 7fb8d158b640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 0x7fb8b006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80099e0 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:50.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.784+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb8bc0169b0 con 0x7fb8cc1d3110 2024-04-02T16:11:50.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8cc0b32b0 con 0x7fb8cc1d3110 2024-04-02T16:11:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8c1ffb640 1 -- 172.21.15.40:0/2122769284 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb8bc0efc50 con 0x7fb8cc1d3110 2024-04-02T16:11:50.941 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 msgr2=0x7fb8b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 0x7fb8b006a840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80099e0 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 msgr2=0x7fb8cc13eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc01cce0 tx=0x7fb8b8005840 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 shutdown_connections 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.940+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8b0068380 0x7fb8b006a840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8cc1e18d0 0x7fb8cc13f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8cc1d3110 0x7fb8cc13eff0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 --2- 172.21.15.40:0/2122769284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8cc142fe0 0x7fb8cc1e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 >> 172.21.15.40:0/2122769284 conn(0x7fb8cc0b8fd0 msgr2=0x7fb8cc1d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 shutdown_connections 2024-04-02T16:11:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:50.944+0000 7fb8d37ef640 1 -- 172.21.15.40:0/2122769284 wait complete. 2024-04-02T16:11:50.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 Processor -- start 2024-04-02T16:11:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 -- start start 2024-04-02T16:11:51.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f82740b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82740b19e0 con 0x7f8274073230 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82740b05c0 con 0x7f82740b23d0 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827bf07640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82740b08a0 con 0x7f8274149560 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8274073230 0x7f8274148e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60782/0 (socket says 172.21.15.40:60782) 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 -- 172.21.15.40:0/3652087956 learned_addr learned my addr 172.21.15.40:0/3652087956 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827a4a4640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f82740b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601401775 0 0) 0x7f82740b19e0 con 0x7f8274073230 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8260003530 con 0x7f8274073230 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 865101538 0 0) 0x7f8260003530 con 0x7f8274073230 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 msgr2=0x7f82740b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f82740b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 >> v1:172.21.15.204:6789/0 conn(0x7f82740b23d0 legacy=0x7f827409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82740b0b80 con 0x7f8274073230 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f8264001680 con 0x7f8274073230 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 >> v1:172.21.15.40:6789/0 conn(0x7f8274073230 legacy=0x7f8274148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82600039f0 0x7f8260003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8260004810 0x7f8260004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f8260005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82740b08a0 con 0x7f8260004810 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f826000bb30 con 0x7f82600039f0 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f826000bd30 con 0x7f8274149560 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8260004810 0x7f8260004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f8279ca3640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82600039f0 0x7f8260003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827a4a4640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f8260005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 -- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 msgr2=0x7f8260005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f8260005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 -- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82600039f0 msgr2=0x7f8260003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82600039f0 0x7f8260003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f82794a2640 1 -- 172.21.15.40:0/3652087956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8260003530 con 0x7f8260004810 2024-04-02T16:11:51.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.076+0000 7f827a4a4640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8274149560 0x7f8260005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f82794a2640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8260004810 0x7f8260004c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f826400c210 tx=0x7f826401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e354d1b21eb64e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264007d40 con 0x7f8260004810 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82640a9070 con 0x7f8260004810 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f8278ca1640 1 -- 172.21.15.40:0/3652087956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264005d40 con 0x7f8260004810 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/3652087956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8260004810 msgr2=0x7f8260004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 --2- 172.21.15.40:0/3652087956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8260004810 0x7f8260004c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f826400c210 tx=0x7f826401cca0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 -- 172.21.15.40:0/3652087956 reap_dead start 2024-04-02T16:11:51.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/3652087956 shutdown_connections 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/3652087956 >> 172.21.15.40:0/3652087956 conn(0x7f82740b8fd0 msgr2=0x7f82740b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/3652087956 shutdown_connections 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/3652087956 wait complete. 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 Processor -- start 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- start start 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8260004810 0x7f827409acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f827409d3e0 0x7f8274139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8274151a20 con 0x7f827409d820 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82741518a0 con 0x7f8260004810 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8274151ba0 con 0x7f827409d3e0 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42812/0 (socket says 172.21.15.40:42812) 2024-04-02T16:11:51.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 -- 172.21.15.40:0/1263354278 learned_addr learned my addr 172.21.15.40:0/1263354278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f8279ca3640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8260004810 0x7f827409acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f82794a2640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f827409d3e0 0x7f8274139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 -- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f827409d3e0 msgr2=0x7f8274139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f827409d3e0 0x7f8274139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 -- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8260004810 msgr2=0x7f827409acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8260004810 0x7f827409acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 -- 172.21.15.40:0/1263354278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827409bf00 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827a4a4640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8274072c70 tx=0x7f826c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c01c070 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c017400 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827409cf50 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0177d0 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.080+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827409b940 con 0x7f827409d820 2024-04-02T16:11:51.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82740685f0 con 0x7f827409d820 2024-04-02T16:11:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f826c01f070 con 0x7f827409d820 2024-04-02T16:11:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f826a7fc640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 0x7f825806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f826c086c60 con 0x7f827409d820 2024-04-02T16:11:51.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f8279ca3640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 0x7f825806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.084+0000 7f8279ca3640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 0x7f825806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8270005490 tx=0x7f8270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.088+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f826c012070 con 0x7f827409d820 2024-04-02T16:11:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.248+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82740b1690 con 0x7f827409d820 2024-04-02T16:11:51.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.248+0000 7f826a7fc640 1 -- 172.21.15.40:0/1263354278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f826c0572b0 con 0x7f827409d820 2024-04-02T16:11:51.399 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 msgr2=0x7f825806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 0x7f825806a840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8270005490 tx=0x7f8270009040 comp rx=0 tx=0).stop 2024-04-02T16:11:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 msgr2=0x7f8274139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8274072c70 tx=0x7f826c00f4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 shutdown_connections 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f827409d3e0 0x7f8274139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8258068380 0x7f825806a840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f827409d820 0x7f8274139560 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 --2- 172.21.15.40:0/1263354278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8260004810 0x7f827409acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 >> 172.21.15.40:0/1263354278 conn(0x7f82740b8fd0 msgr2=0x7f827406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 shutdown_connections 2024-04-02T16:11:51.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.252+0000 7f827bf07640 1 -- 172.21.15.40:0/1263354278 wait complete. 2024-04-02T16:11:51.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:51.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 Processor -- start 2024-04-02T16:11:51.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 -- start start 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e80b1c10 0x7fa7e80b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e80b0a10 con 0x7fa7e80ae080 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e80b3210 con 0x7fa7e80b25b0 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7ed71a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e80b2a00 con 0x7fa7e80b1c10 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e80b1c10 0x7fa7e80b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa7e80ae080 0x7fa7e80b0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60784/0 (socket says 172.21.15.40:60784) 2024-04-02T16:11:51.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e6ffd640 1 -- 172.21.15.40:0/219854141 learned_addr learned my addr 172.21.15.40:0/219854141 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:51.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e67fc640 1 -- 172.21.15.40:0/219854141 >> v1:172.21.15.204:6789/0 conn(0x7fa7e80b25b0 legacy=0x7fa7e81522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e67fc640 1 -- 172.21.15.40:0/219854141 >> v1:172.21.15.40:6789/0 conn(0x7fa7e80ae080 legacy=0x7fa7e80b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e5ffb640 1 -- 172.21.15.40:0/219854141 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070909271 0 0) 0x7fa7e80b0a10 con 0x7fa7e80ae080 2024-04-02T16:11:51.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.528+0000 7fa7e67fc640 1 -- 172.21.15.40:0/219854141 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e806db10 con 0x7fa7e80b1c10 2024-04-02T16:11:51.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e67fc640 1 --2- 172.21.15.40:0/219854141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b1c10 0x7fa7e80b2070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2b80 tx=0x7fa7d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b9432414eb3987 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e5ffb640 1 -- 172.21.15.40:0/219854141 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80b1070 con 0x7fa7e80b1c10 2024-04-02T16:11:51.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e5ffb640 1 -- 172.21.15.40:0/219854141 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d80ac5f0 con 0x7fa7e80b1c10 2024-04-02T16:11:51.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e5ffb640 1 -- 172.21.15.40:0/219854141 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80ac9c0 con 0x7fa7e80b1c10 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- 172.21.15.40:0/219854141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b1c10 msgr2=0x7fa7e80b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/219854141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b1c10 0x7fa7e80b2070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2b80 tx=0x7fa7d801c4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- 172.21.15.40:0/219854141 shutdown_connections 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/219854141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80b1c10 0x7fa7e80b2070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- 172.21.15.40:0/219854141 >> 172.21.15.40:0/219854141 conn(0x7fa7e80b8fd0 msgr2=0x7fa7e80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- 172.21.15.40:0/219854141 shutdown_connections 2024-04-02T16:11:51.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- 172.21.15.40:0/219854141 wait complete. 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 Processor -- start 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- start start 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 0x7fa7e809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80b25b0 0x7fa7e8070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80ae780 con 0x7fa7e80ae080 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80afe60 con 0x7fa7e80b1c10 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7ed71a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80af2f0 con 0x7fa7e80b25b0 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42832/0 (socket says 172.21.15.40:42832) 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 -- 172.21.15.40:0/2675269341 learned_addr learned my addr 172.21.15.40:0/2675269341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e77fe640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80b25b0 0x7fa7e8070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 -- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80b25b0 msgr2=0x7fa7e8070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80b25b0 0x7fa7e8070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e67fc640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 0x7fa7e809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 -- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 msgr2=0x7fa7e809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 0x7fa7e809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e6ffd640 1 -- 172.21.15.40:0/2675269341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e809afb0 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.532+0000 7fa7e67fc640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 0x7fa7e809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7e6ffd640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00c4a0 tx=0x7fa7dc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc018070 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0074e0 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e8070f00 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc004a30 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e809c6d0 con 0x7fa7e80ae080 2024-04-02T16:11:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e80685f0 con 0x7fa7e80ae080 2024-04-02T16:11:51.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa7dc01b070 con 0x7fa7e80ae080 2024-04-02T16:11:51.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 0x7fa7d406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa7dc085be0 con 0x7fa7e80ae080 2024-04-02T16:11:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7e67fc640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 0x7fa7d406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.536+0000 7fa7e67fc640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 0x7fa7d406a870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ae000 tx=0x7fa7d80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.540+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa7e8070f00 con 0x7fa7e80ae080 2024-04-02T16:11:51.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.696+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7e80b6c40 con 0x7fa7e80ae080 2024-04-02T16:11:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.700+0000 7fa7cf7fe640 1 -- 172.21.15.40:0/2675269341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fa7dc056230 con 0x7fa7e80ae080 2024-04-02T16:11:51.701 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:51.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.700+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 msgr2=0x7fa7d406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.700+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 0x7fa7d406a870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ae000 tx=0x7fa7d80a7000 comp rx=0 tx=0).stop 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.700+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 msgr2=0x7fa7e814d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.700+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00c4a0 tx=0x7fa7dc00c960 comp rx=0 tx=0).stop 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 shutdown_connections 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7e80b25b0 0x7fa7e8070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa7d40683b0 0x7fa7d406a870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7e80b1c10 0x7fa7e809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 --2- 172.21.15.40:0/2675269341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e80ae080 0x7fa7e814d610 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 >> 172.21.15.40:0/2675269341 conn(0x7fa7e80b8fd0 msgr2=0x7fa7e81532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 shutdown_connections 2024-04-02T16:11:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.704+0000 7fa7ed71a640 1 -- 172.21.15.40:0/2675269341 wait complete. 2024-04-02T16:11:51.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:51.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:51.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 Processor -- start 2024-04-02T16:11:51.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 -- start start 2024-04-02T16:11:51.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77fc151830 0x7f77fc153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f77fc154160 0x7f77fc156550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77fc0af670 con 0x7f77fc0ae320 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f77fc0af950 con 0x7f77fc151830 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f780201f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f77fc0aff10 con 0x7f77fc154160 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77fc151830 0x7f77fc153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77fc151830 0x7f77fc153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42840/0 (socket says 172.21.15.40:42840) 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 -- 172.21.15.40:0/675988881 learned_addr learned my addr 172.21.15.40:0/675988881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77fbfff640 1 --2- 172.21.15.40:0/675988881 >> v2:172.21.15.204:3300/0 conn(0x7f77fc154160 0x7f77fc156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 -- 172.21.15.40:0/675988881 >> v2:172.21.15.204:3300/0 conn(0x7f77fc154160 msgr2=0x7f77fc156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 --2- 172.21.15.40:0/675988881 >> v2:172.21.15.204:3300/0 conn(0x7f77fc154160 0x7f77fc156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 -- 172.21.15.40:0/675988881 >> v1:172.21.15.204:6789/0 conn(0x7f77fc0ae320 legacy=0x7f77fc06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 -- 172.21.15.40:0/675988881 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc0b5260 con 0x7f77fc151830 2024-04-02T16:11:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77faffd640 1 --2- 172.21.15.40:0/675988881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc153c20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0b4140 tx=0x7f77e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c007c73efd37e907 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77fa7fc640 1 -- 172.21.15.40:0/675988881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40b0070 con 0x7f77fc151830 2024-04-02T16:11:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77fa7fc640 1 -- 172.21.15.40:0/675988881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e40ab600 con 0x7f77fc151830 2024-04-02T16:11:51.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.852+0000 7f77fa7fc640 1 -- 172.21.15.40:0/675988881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40ab9d0 con 0x7f77fc151830 2024-04-02T16:11:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/675988881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 msgr2=0x7f77fc153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- 172.21.15.40:0/675988881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc153c20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0b4140 tx=0x7f77e401c3d0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/675988881 shutdown_connections 2024-04-02T16:11:51.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- 172.21.15.40:0/675988881 >> v2:172.21.15.204:3300/0 conn(0x7f77fc154160 0x7f77fc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- 172.21.15.40:0/675988881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc153c20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/675988881 >> 172.21.15.40:0/675988881 conn(0x7f77fc0b8fd0 msgr2=0x7f77fc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/675988881 shutdown_connections 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/675988881 wait complete. 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 Processor -- start 2024-04-02T16:11:51.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- start start 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 0x7f77fc14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc154160 0x7f77fc1e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0af2f0 con 0x7f77fc151830 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc0b0300 con 0x7f77fc154160 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f77fc0aeef0 con 0x7f77fc0ae320 2024-04-02T16:11:51.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42852/0 (socket says 172.21.15.40:42852) 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 -- 172.21.15.40:0/1980716407 learned_addr learned my addr 172.21.15.40:0/1980716407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77fb7fe640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 0x7f77fc14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 -- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 msgr2=0x7f77fc14b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 0x7f77fc14b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 -- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc154160 msgr2=0x7f77fc1e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc154160 0x7f77fc1e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 -- 172.21.15.40:0/1980716407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc1548c0 con 0x7f77fc151830 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77fb7fe640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 0x7f77fc14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77faffd640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77e40ae040 tx=0x7f77e401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40ba020 con 0x7f77fc151830 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e40b6620 con 0x7f77fc151830 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc155260 con 0x7f77fc151830 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e40b0040 con 0x7f77fc151830 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.856+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc154540 con 0x7f77fc151830 2024-04-02T16:11:51.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77fc0685f0 con 0x7f77fc151830 2024-04-02T16:11:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f77e40bd020 con 0x7f77fc151830 2024-04-02T16:11:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f77f3fff640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 0x7f77dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77e4120d10 con 0x7f77fc151830 2024-04-02T16:11:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f77fb7fe640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 0x7f77dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.860+0000 7f77fb7fe640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 0x7f77dc06a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007fd0 tx=0x7f77f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:51.864+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f77e4126020 con 0x7f77fc151830 2024-04-02T16:11:52.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77fc152090 con 0x7f77fc151830 2024-04-02T16:11:52.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f77f3fff640 1 -- 172.21.15.40:0/1980716407 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f77e40c4030 con 0x7f77fc151830 2024-04-02T16:11:52.028 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 msgr2=0x7f77dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 0x7f77dc06a840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007fd0 tx=0x7f77f4009040 comp rx=0 tx=0).stop 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 msgr2=0x7f77fc0b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77e40ae040 tx=0x7f77e401ac10 comp rx=0 tx=0).stop 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 shutdown_connections 2024-04-02T16:11:52.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f77dc068380 0x7f77dc06a840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77fc154160 0x7f77fc1e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77fc151830 0x7f77fc0b1fc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 --2- 172.21.15.40:0/1980716407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77fc0ae320 0x7f77fc14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 >> 172.21.15.40:0/1980716407 conn(0x7f77fc0b8fd0 msgr2=0x7f77fc06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 shutdown_connections 2024-04-02T16:11:52.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.028+0000 7f780201f640 1 -- 172.21.15.40:0/1980716407 wait complete. 2024-04-02T16:11:52.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:52.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 Processor -- start 2024-04-02T16:11:52.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 -- start start 2024-04-02T16:11:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8e00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e00b19e0 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e00b05c0 con 0x7fb8e00b23d0 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8e4a86640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00b08a0 con 0x7fb8e0073230 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8ded9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb8e0148e50 0x7fb8e01492c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60786/0 (socket says 172.21.15.40:60786) 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.176+0000 7fb8ded9d640 1 -- 172.21.15.40:0/1330319837 learned_addr learned my addr 172.21.15.40:0/1330319837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8e00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457635021 0 0) 0x7fb8e00b19e0 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8c8003530 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 823979981 0 0) 0x7fb8c8003530 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 msgr2=0x7fb8e00b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8e00b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 >> v1:172.21.15.204:6789/0 conn(0x7fb8e00b23d0 legacy=0x7fb8e009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00b0b80 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fb8d8001a00 con 0x7fb8e0148e50 2024-04-02T16:11:52.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 >> v1:172.21.15.40:6789/0 conn(0x7fb8e0148e50 legacy=0x7fb8e01492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c80039f0 0x7fb8c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c8004810 0x7fb8c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b08a0 con 0x7fb8c80039f0 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c800bb30 con 0x7fb8c8004810 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c800bd30 con 0x7fb8e0073230 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c80039f0 0x7fb8c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c8004810 0x7fb8c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8ded9d640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 -- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 msgr2=0x7fb8c8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e0073230 0x7fb8c8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 -- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c8004810 msgr2=0x7fb8c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c8004810 0x7fb8c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 -- 172.21.15.40:0/1330319837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c8003530 con 0x7fb8c80039f0 2024-04-02T16:11:52.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c8004810 0x7fb8c8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c80039f0 0x7fb8c8003e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001780 tx=0x7fb8d001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308532a5af6c0968 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00013a0 con 0x7fb8c80039f0 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d00052f0 con 0x7fb8c80039f0 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.180+0000 7fb8dd59a640 1 -- 172.21.15.40:0/1330319837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003840 con 0x7fb8c80039f0 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1330319837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c80039f0 msgr2=0x7fb8c8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1330319837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8c80039f0 0x7fb8c8003e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001780 tx=0x7fb8d001c420 comp rx=0 tx=0).stop 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ded9d640 1 -- 172.21.15.40:0/1330319837 reap_dead start 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1330319837 shutdown_connections 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1330319837 >> 172.21.15.40:0/1330319837 conn(0x7fb8e00b8fd0 msgr2=0x7fb8e00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1330319837 shutdown_connections 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1330319837 wait complete. 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 Processor -- start 2024-04-02T16:11:52.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- start start 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c80039f0 0x7fb8e014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e014b8c0 0x7fb8e013d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0151a20 con 0x7fb8e01e4b50 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01518a0 con 0x7fb8c80039f0 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e0151ba0 con 0x7fb8e014b8c0 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42874/0 (socket says 172.21.15.40:42874) 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 -- 172.21.15.40:0/1176001649 learned_addr learned my addr 172.21.15.40:0/1176001649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c80039f0 0x7fb8e014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ded9d640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e014b8c0 0x7fb8e013d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 -- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e014b8c0 msgr2=0x7fb8e013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e014b8c0 0x7fb8e013d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 -- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c80039f0 msgr2=0x7fb8e014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c80039f0 0x7fb8e014de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 -- 172.21.15.40:0/1176001649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e013ef50 con 0x7fb8e01e4b50 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ddd9b640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4011700 tx=0x7fb8d4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4004220 con 0x7fb8e01e4b50 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d40043c0 con 0x7fb8e01e4b50 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e009b690 con 0x7fb8e01e4b50 2024-04-02T16:11:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400f960 con 0x7fb8e01e4b50 2024-04-02T16:11:52.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.184+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01e4350 con 0x7fb8e01e4b50 2024-04-02T16:11:52.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e00685f0 con 0x7fb8e01e4b50 2024-04-02T16:11:52.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb8d400fb00 con 0x7fb8e01e4b50 2024-04-02T16:11:52.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8ceffd640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d4086ef0 con 0x7fb8e01e4b50 2024-04-02T16:11:52.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.188+0000 7fb8de59c640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 0x7fb8c006a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001750 tx=0x7fb8d00011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.192+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb8d4019030 con 0x7fb8e01e4b50 2024-04-02T16:11:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.352+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8e01488e0 con 0x7fb8e01e4b50 2024-04-02T16:11:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8ceffd640 1 -- 172.21.15.40:0/1176001649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb8d4056e50 con 0x7fb8e01e4b50 2024-04-02T16:11:52.356 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 msgr2=0x7fb8c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 0x7fb8c006a840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001750 tx=0x7fb8d00011a0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 msgr2=0x7fb8e014b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4011700 tx=0x7fb8d4011bc0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 shutdown_connections 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8e014b8c0 0x7fb8e013d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb8c0068380 0x7fb8c006a840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8e01e4b50 0x7fb8e014b380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 --2- 172.21.15.40:0/1176001649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8c80039f0 0x7fb8e014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 >> 172.21.15.40:0/1176001649 conn(0x7fb8e00b8fd0 msgr2=0x7fb8e00b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 shutdown_connections 2024-04-02T16:11:52.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.356+0000 7fb8e4a86640 1 -- 172.21.15.40:0/1176001649 wait complete. 2024-04-02T16:11:52.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:52.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 Processor -- start 2024-04-02T16:11:52.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 -- start start 2024-04-02T16:11:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c340b3500 0x7f0c340b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 0x7f0c340b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340b6310 con 0x7f0c340b4bf0 2024-04-02T16:11:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340b55d0 con 0x7f0c340b3500 2024-04-02T16:11:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c3a11a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3414b560 con 0x7f0c340b8c80 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c337fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0c340b4bf0 0x7f0c3414ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60794/0 (socket says 172.21.15.40:60794) 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c337fe640 1 -- 172.21.15.40:0/1540857533 learned_addr learned my addr 172.21.15.40:0/1540857533 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c33fff640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 0x7f0c340b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/1540857533 >> v2:172.21.15.204:3300/0 conn(0x7f0c340b3500 0x7f0c340b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195873544 0 0) 0x7f0c340b6310 con 0x7f0c340b4bf0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c20003530 con 0x7f0c340b4bf0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1191756734 0 0) 0x7f0c20003530 con 0x7f0c340b4bf0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 msgr2=0x7f0c340b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 0x7f0c340b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 >> v2:172.21.15.204:3300/0 conn(0x7f0c340b3500 msgr2=0x7f0c340b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 --2- 172.21.15.40:0/1540857533 >> v2:172.21.15.204:3300/0 conn(0x7f0c340b3500 0x7f0c340b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340b3fa0 con 0x7f0c340b4bf0 2024-04-02T16:11:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f0c2c001a00 con 0x7f0c340b4bf0 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 >> v1:172.21.15.40:6789/0 conn(0x7f0c340b4bf0 legacy=0x7f0c3414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 0x7f0c200039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c20003f90 0x7f0c20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20004d50 0x7f0c200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c340b55d0 con 0x7f0c20003f90 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3414b560 con 0x7f0c20004d50 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2000bb00 con 0x7f0c340b8c80 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c20003f90 0x7f0c20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c33fff640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20004d50 0x7f0c200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 -- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 msgr2=0x7f0c200039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c340b8c80 0x7f0c200039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 -- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20004d50 msgr2=0x7f0c200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20004d50 0x7f0c200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c32ffd640 1 -- 172.21.15.40:0/1540857533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c20003530 con 0x7f0c20003f90 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.500+0000 7f0c33fff640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20004d50 0x7f0c200051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c20003f90 0x7f0c20004810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ae0 tx=0x7f0c1c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2a91e7c0fab551 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0096b0 con 0x7f0c20003f90 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0a6490 con 0x7f0c20003f90 2024-04-02T16:11:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c327fc640 1 -- 172.21.15.40:0/1540857533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0a6880 con 0x7f0c20003f90 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- 172.21.15.40:0/1540857533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c20003f90 msgr2=0x7f0c20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/1540857533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c20003f90 0x7f0c20004810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009ae0 tx=0x7f0c1c01cab0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c33fff640 1 -- 172.21.15.40:0/1540857533 reap_dead start 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- 172.21.15.40:0/1540857533 shutdown_connections 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- 172.21.15.40:0/1540857533 >> 172.21.15.40:0/1540857533 conn(0x7f0c340734b0 msgr2=0x7f0c340bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- 172.21.15.40:0/1540857533 shutdown_connections 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- 172.21.15.40:0/1540857533 wait complete. 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 Processor -- start 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- start start 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20003f90 0x7f0c34144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c341e3a60 0x7f0c341e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34152c00 con 0x7f0c341e0710 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34152d80 con 0x7f0c20003f90 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c3a11a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0c34152f00 con 0x7f0c341e3a60 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42898/0 (socket says 172.21.15.40:42898) 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 -- 172.21.15.40:0/2508042724 learned_addr learned my addr 172.21.15.40:0/2508042724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c337fe640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20003f90 0x7f0c34144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c33fff640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c341e3a60 0x7f0c341e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 -- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c341e3a60 msgr2=0x7f0c341e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c341e3a60 0x7f0c341e3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 -- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20003f90 msgr2=0x7f0c34144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20003f90 0x7f0c34144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 -- 172.21.15.40:0/2508042724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341e10d0 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.504+0000 7f0c32ffd640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c01c8e0 tx=0x7f0c20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0be070 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0a7470 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34144840 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0096b0 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c34142ee0 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34148280 con 0x7f0c341e0710 2024-04-02T16:11:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0c1c0c1070 con 0x7f0c341e0710 2024-04-02T16:11:52.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 0x7f0c1406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c120970 con 0x7f0c341e0710 2024-04-02T16:11:52.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c337fe640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 0x7f0c1406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.508+0000 7f0c337fe640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 0x7f0c1406a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007fd0 tx=0x7f0c2c007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.512+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f0c34148280 con 0x7f0c341e0710 2024-04-02T16:11:52.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.676+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0c340685f0 con 0x7f0c341e0710 2024-04-02T16:11:52.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.676+0000 7f0c2bfff640 1 -- 172.21.15.40:0/2508042724 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0c341e10d0 con 0x7f0c341e0710 2024-04-02T16:11:52.679 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:52.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 msgr2=0x7f0c1406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 0x7f0c1406a840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007fd0 tx=0x7f0c2c007d60 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 msgr2=0x7f0c341e0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c01c8e0 tx=0x7f0c20005840 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 shutdown_connections 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c341e3a60 0x7f0c341e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0c14068380 0x7f0c1406a840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c341e0710 0x7f0c341e0b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 --2- 172.21.15.40:0/2508042724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c20003f90 0x7f0c34144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 >> 172.21.15.40:0/2508042724 conn(0x7f0c340734b0 msgr2=0x7f0c340ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 shutdown_connections 2024-04-02T16:11:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.680+0000 7f0c3a11a640 1 -- 172.21.15.40:0/2508042724 wait complete. 2024-04-02T16:11:52.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 Processor -- start 2024-04-02T16:11:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 -- start start 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f61dc09e110 0x7f61dc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61dc143d10 con 0x7f61dc09dcb0 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61dc06d510 con 0x7f61dc06d130 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e26e6640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f61dc06cd50 con 0x7f61dc09e110 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61d3fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61dc09dcb0 0x7f61dc144520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60810/0 (socket says 172.21.15.40:60810) 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61d3fff640 1 -- 172.21.15.40:0/254650520 learned_addr learned my addr 172.21.15.40:0/254650520 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 --2- 172.21.15.40:0/254650520 >> v2:172.21.15.204:3300/0 conn(0x7f61dc09e110 0x7f61dc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2160524914 0 0) 0x7f61dc143d10 con 0x7f61dc09dcb0 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c8003530 con 0x7f61dc09dcb0 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 879707989 0 0) 0x7f61c8003530 con 0x7f61dc09dcb0 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 >> v2:172.21.15.204:3300/0 conn(0x7f61dc09e110 msgr2=0x7f61dc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 --2- 172.21.15.40:0/254650520 >> v2:172.21.15.204:3300/0 conn(0x7f61dc09e110 0x7f61dc0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 >> v1:172.21.15.204:6789/0 conn(0x7f61dc06d130 legacy=0x7f61dc06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc0aea10 con 0x7f61dc09dcb0 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f61c4001680 con 0x7f61dc09dcb0 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 >> v1:172.21.15.40:6789/0 conn(0x7f61dc09dcb0 legacy=0x7f61dc144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc06cd50 con 0x7f61c80051d0 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61c800c170 con 0x7f61c80039c0 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f61c800c370 con 0x7f61c8004810 2024-04-02T16:11:52.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61dbfff640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61d3fff640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 -- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c8004810 msgr2=0x7f61c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61c8004810 0x7f61c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 -- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c80039c0 msgr2=0x7f61c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61c80039c0 0x7f61c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.816+0000 7f61e0c83640 1 -- 172.21.15.40:0/254650520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c8003530 con 0x7f61c80051d0 2024-04-02T16:11:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e0c83640 1 -- 172.21.15.40:0/254650520 reap_dead start 2024-04-02T16:11:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e0c83640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61c800b9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009f90 tx=0x7f61d401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2011d6e9e6224030 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40b2070 con 0x7f61c80051d0 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d40a5b80 con 0x7f61c80051d0 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61db7fe640 1 -- 172.21.15.40:0/254650520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4009c10 con 0x7f61c80051d0 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 msgr2=0x7f61c800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61c800b9d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009f90 tx=0x7f61d401cc80 comp rx=0 tx=0).stop 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- 172.21.15.40:0/254650520 shutdown_connections 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 --2- 172.21.15.40:0/254650520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61c800b9d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- 172.21.15.40:0/254650520 >> 172.21.15.40:0/254650520 conn(0x7f61dc0b8fd0 msgr2=0x7f61dc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- 172.21.15.40:0/254650520 shutdown_connections 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- 172.21.15.40:0/254650520 wait complete. 2024-04-02T16:11:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 Processor -- start 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- start start 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61dc09c4d0 0x7f61dc13efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 0x7f61dc0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0af060 con 0x7f61c80051d0 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0ae7c0 con 0x7f61dc09c4d0 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61e26e6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc0ae300 con 0x7f61dc13f4e0 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42918/0 (socket says 172.21.15.40:42918) 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.820+0000 7f61dbfff640 1 -- 172.21.15.40:0/3651843434 learned_addr learned my addr 172.21.15.40:0/3651843434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61e0c83640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 0x7f61dc0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d3fff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61dc09c4d0 0x7f61dc13efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 -- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 msgr2=0x7f61dc0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 0x7f61dc0708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 -- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61dc09c4d0 msgr2=0x7f61dc13efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61dc09c4d0 0x7f61dc13efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 -- 172.21.15.40:0/3651843434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc09c1f0 con 0x7f61c80051d0 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61e0c83640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 0x7f61dc0708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61dbfff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e3f0 tx=0x7f61cc00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc019070 con 0x7f61c80051d0 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc0052e0 con 0x7f61c80051d0 2024-04-02T16:11:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1ca110 con 0x7f61c80051d0 2024-04-02T16:11:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003820 con 0x7f61c80051d0 2024-04-02T16:11:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1c9d90 con 0x7f61c80051d0 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc0685f0 con 0x7f61c80051d0 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f61cc01c070 con 0x7f61c80051d0 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 0x7f61bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61cc085e90 con 0x7f61c80051d0 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d3fff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 0x7f61bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.824+0000 7f61d3fff640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 0x7f61bc06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001060 tx=0x7f61c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.828+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f61cc055e90 con 0x7f61c80051d0 2024-04-02T16:11:52.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:52.996+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61dc09d830 con 0x7f61c80051d0 2024-04-02T16:11:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61d97fa640 1 -- 172.21.15.40:0/3651843434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f61cc05a190 con 0x7f61c80051d0 2024-04-02T16:11:53.001 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:53.001 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 msgr2=0x7f61bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 0x7f61bc06a840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61c4001060 tx=0x7f61c4009040 comp rx=0 tx=0).stop 2024-04-02T16:11:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 msgr2=0x7f61dc1dc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00e3f0 tx=0x7f61cc00e8b0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 shutdown_connections 2024-04-02T16:11:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61dc13f4e0 0x7f61dc0708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f61bc068380 0x7f61bc06a840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61dc09c4d0 0x7f61dc13efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 --2- 172.21.15.40:0/3651843434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61c80051d0 0x7f61dc1dc270 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 >> 172.21.15.40:0/3651843434 conn(0x7f61dc0b8fd0 msgr2=0x7f61dc141340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 shutdown_connections 2024-04-02T16:11:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.000+0000 7f61e26e6640 1 -- 172.21.15.40:0/3651843434 wait complete. 2024-04-02T16:11:53.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:53.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.144+0000 7fc00a95a640 1 Processor -- start 2024-04-02T16:11:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.144+0000 7fc00a95a640 1 -- start start 2024-04-02T16:11:53.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc00a95a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc004073230 0x7fc0040b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc00a95a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004148e50 0x7fc004149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc00a95a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0040b19e0 con 0x7fc0040b23d0 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc00a95a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc0041497c0 con 0x7fc004073230 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc00a95a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc0040b05c0 con 0x7fc004148e50 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc003fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc0040b23d0 0x7fc00409daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60820/0 (socket says 172.21.15.40:60820) 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc003fff640 1 -- 172.21.15.40:0/3897456300 learned_addr learned my addr 172.21.15.40:0/3897456300 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 --2- 172.21.15.40:0/3897456300 >> v2:172.21.15.40:3300/0 conn(0x7fc004073230 0x7fc0040b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc008ef7640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004148e50 0x7fc004149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc002ffd640 1 -- 172.21.15.40:0/3897456300 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 926611156 0 0) 0x7fc0040b19e0 con 0x7fc0040b23d0 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc002ffd640 1 -- 172.21.15.40:0/3897456300 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfec003530 con 0x7fc0040b23d0 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 -- 172.21.15.40:0/3897456300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004148e50 msgr2=0x7fc004149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004148e50 0x7fc004149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 -- 172.21.15.40:0/3897456300 >> v1:172.21.15.40:6789/0 conn(0x7fc0040b23d0 legacy=0x7fc00409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 -- 172.21.15.40:0/3897456300 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0040b08a0 con 0x7fc004073230 2024-04-02T16:11:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc0037fe640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004073230 0x7fc0040b27b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbff80097c0 tx=0x7fbff801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82d60309559fd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc002ffd640 1 -- 172.21.15.40:0/3897456300 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80ab430 con 0x7fc004073230 2024-04-02T16:11:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc002ffd640 1 -- 172.21.15.40:0/3897456300 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff80b0070 con 0x7fc004073230 2024-04-02T16:11:53.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.148+0000 7fc002ffd640 1 -- 172.21.15.40:0/3897456300 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80ab7f0 con 0x7fc004073230 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/3897456300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004073230 msgr2=0x7fc0040b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004073230 0x7fc0040b27b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbff80097c0 tx=0x7fbff801c410 comp rx=0 tx=0).stop 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/3897456300 shutdown_connections 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004148e50 0x7fc004149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- 172.21.15.40:0/3897456300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004073230 0x7fc0040b27b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/3897456300 >> 172.21.15.40:0/3897456300 conn(0x7fc0040b8fd0 msgr2=0x7fc0040b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/3897456300 shutdown_connections 2024-04-02T16:11:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/3897456300 wait complete. 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 Processor -- start 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- start start 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004073230 0x7fc0041438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0040b23d0 0x7fc00413a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc004152810 con 0x7fc004148e50 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc004152690 con 0x7fc0040b23d0 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc004152990 con 0x7fc004073230 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42934/0 (socket says 172.21.15.40:42934) 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 -- 172.21.15.40:0/553448426 learned_addr learned my addr 172.21.15.40:0/553448426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc0037fe640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0040b23d0 0x7fc00413a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 -- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004073230 msgr2=0x7fc0041438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004073230 0x7fc0041438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 -- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0040b23d0 msgr2=0x7fc00413a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0040b23d0 0x7fc00413a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 -- 172.21.15.40:0/553448426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0041de470 con 0x7fc004148e50 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc008ef7640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007e60 tx=0x7fbffc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc014420 con 0x7fc004148e50 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc019070 con 0x7fc004148e50 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0041de750 con 0x7fc004148e50 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc014820 con 0x7fc004148e50 2024-04-02T16:11:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0041462b0 con 0x7fc004148e50 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.152+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0040685f0 con 0x7fc004148e50 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.156+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbffc0149c0 con 0x7fc004148e50 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.156+0000 7fc000ff9640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 0x7fbff006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.156+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbffc0871b0 con 0x7fc004148e50 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.156+0000 7fc003fff640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 0x7fbff006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.156+0000 7fc003fff640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 0x7fbff006a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbff4007fd0 tx=0x7fbff4007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.160+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fbffc057760 con 0x7fc004148e50 2024-04-02T16:11:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.320+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0040b1690 con 0x7fc004148e50 2024-04-02T16:11:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc000ff9640 1 -- 172.21.15.40:0/553448426 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fbffc057060 con 0x7fc004148e50 2024-04-02T16:11:53.324 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 msgr2=0x7fbff006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 0x7fbff006a840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbff4007fd0 tx=0x7fbff4007900 comp rx=0 tx=0).stop 2024-04-02T16:11:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 msgr2=0x7fc0041453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007e60 tx=0x7fbffc00e4b0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 shutdown_connections 2024-04-02T16:11:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbff0068380 0x7fbff006a840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc004148e50 0x7fc0041453d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0040b23d0 0x7fc00413a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 --2- 172.21.15.40:0/553448426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc004073230 0x7fc0041438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 >> 172.21.15.40:0/553448426 conn(0x7fc0040b8fd0 msgr2=0x7fc00406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 shutdown_connections 2024-04-02T16:11:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.324+0000 7fc00a95a640 1 -- 172.21.15.40:0/553448426 wait complete. 2024-04-02T16:11:53.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 Processor -- start 2024-04-02T16:11:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 -- start start 2024-04-02T16:11:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efd7c148370 0x7efd7c0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd7c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c06d120 con 0x7efd7c06e1d0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efd7c148750 con 0x7efd7c148370 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd8305c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c06d820 con 0x7efd7c0b7510 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd815f9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd7c06e1d0 0x7efd7c0719f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60828/0 (socket says 172.21.15.40:60828) 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd815f9640 1 -- 172.21.15.40:0/2794754638 learned_addr learned my addr 172.21.15.40:0/2794754638 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd7c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd80df8640 1 --2- 172.21.15.40:0/2794754638 >> v2:172.21.15.204:3300/0 conn(0x7efd7c148370 0x7efd7c0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844134434 0 0) 0x7efd7c06d120 con 0x7efd7c06e1d0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003530 con 0x7efd7c06e1d0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1992024587 0 0) 0x7efd64003530 con 0x7efd7c06e1d0 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 msgr2=0x7efd7c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd7c06dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 >> v2:172.21.15.204:3300/0 conn(0x7efd7c148370 msgr2=0x7efd7c0b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 --2- 172.21.15.40:0/2794754638 >> v2:172.21.15.204:3300/0 conn(0x7efd7c148370 0x7efd7c0b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c147770 con 0x7efd7c06e1d0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7efd74001a00 con 0x7efd7c06e1d0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 >> v1:172.21.15.40:6789/0 conn(0x7efd7c06e1d0 legacy=0x7efd7c0719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640039c0 0x7efd64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd64004810 0x7efd64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c148750 con 0x7efd64004810 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400b9d0 con 0x7efd640039c0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd6400bb80 con 0x7efd7c0b7510 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd64004810 0x7efd64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd80df8640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640039c0 0x7efd64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd815f9640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 -- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 msgr2=0x7efd640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 -- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640039c0 msgr2=0x7efd64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd640039c0 0x7efd64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd7bfff640 1 -- 172.21.15.40:0/2794754638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd64003530 con 0x7efd64004810 2024-04-02T16:11:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.472+0000 7efd815f9640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd7c0b7510 0x7efd640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd7bfff640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd64004810 0x7efd64004c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd70009a90 tx=0x7efd7001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8081f2d9a4d7cebb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70009690 con 0x7efd64004810 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd700a9430 con 0x7efd64004810 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd7b7fe640 1 -- 172.21.15.40:0/2794754638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700a9800 con 0x7efd64004810 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- 172.21.15.40:0/2794754638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd64004810 msgr2=0x7efd64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 --2- 172.21.15.40:0/2794754638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd64004810 0x7efd64004c90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd70009a90 tx=0x7efd7001c9d0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd815f9640 1 -- 172.21.15.40:0/2794754638 reap_dead start 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- 172.21.15.40:0/2794754638 shutdown_connections 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- 172.21.15.40:0/2794754638 >> 172.21.15.40:0/2794754638 conn(0x7efd7c0bb640 msgr2=0x7efd7c0b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- 172.21.15.40:0/2794754638 shutdown_connections 2024-04-02T16:11:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- 172.21.15.40:0/2794754638 wait complete. 2024-04-02T16:11:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 Processor -- start 2024-04-02T16:11:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.476+0000 7efd8305c640 1 -- start start 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd64004810 0x7efd7c14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 0x7efd7c1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0b8700 con 0x7efd7c149710 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0b59b0 con 0x7efd7c14a7e0 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c0b8880 con 0x7efd64004810 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42946/0 (socket says 172.21.15.40:42946) 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 -- 172.21.15.40:0/3082149038 learned_addr learned my addr 172.21.15.40:0/3082149038 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd80df8640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd64004810 0x7efd7c14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd7bfff640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 0x7efd7c1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 -- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd64004810 msgr2=0x7efd7c14b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd64004810 0x7efd7c14b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 -- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 msgr2=0x7efd7c1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 0x7efd7c1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 -- 172.21.15.40:0/3082149038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c148e90 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd7bfff640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 0x7efd7c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd815f9640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd74000f90 tx=0x7efd7400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74004d70 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7400bb80 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c14acc0 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74007110 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1d7900 con 0x7efd7c149710 2024-04-02T16:11:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c068d00 con 0x7efd7c149710 2024-04-02T16:11:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efd7401b020 con 0x7efd7c149710 2024-04-02T16:11:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 0x7efd6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.480+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd740873b0 con 0x7efd7c149710 2024-04-02T16:11:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.484+0000 7efd80df8640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 0x7efd6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.484+0000 7efd80df8640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 0x7efd6806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd7c1d6620 tx=0x7efd6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.488+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7efd74022070 con 0x7efd7c149710 2024-04-02T16:11:53.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efd7c06e680 con 0x7efd7c149710 2024-04-02T16:11:53.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd797fa640 1 -- 172.21.15.40:0/3082149038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7efd74057960 con 0x7efd7c149710 2024-04-02T16:11:53.653 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:53.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 msgr2=0x7efd6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 0x7efd6806a840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efd7c1d6620 tx=0x7efd6c008040 comp rx=0 tx=0).stop 2024-04-02T16:11:53.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 msgr2=0x7efd7c1e3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd74000f90 tx=0x7efd7400b4a0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 shutdown_connections 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd68068380 0x7efd6806a840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd7c149710 0x7efd7c1e3470 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd7c14a7e0 0x7efd7c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 --2- 172.21.15.40:0/3082149038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd64004810 0x7efd7c14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 >> 172.21.15.40:0/3082149038 conn(0x7efd7c0bb640 msgr2=0x7efd7c140450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 shutdown_connections 2024-04-02T16:11:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.652+0000 7efd8305c640 1 -- 172.21.15.40:0/3082149038 wait complete. 2024-04-02T16:11:53.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:53.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.800+0000 7f65644ae640 1 Processor -- start 2024-04-02T16:11:53.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.800+0000 7f65644ae640 1 -- start start 2024-04-02T16:11:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c073230 0x7f655c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c0b19e0 con 0x7f655c0b23d0 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f655c1497c0 con 0x7f655c073230 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0b05c0 con 0x7f655c148e50 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c073230 0x7f655c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f655c073230 0x7f655c0b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42952/0 (socket says 172.21.15.40:42952) 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 -- 172.21.15.40:0/182378553 learned_addr learned my addr 172.21.15.40:0/182378553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6562a4b640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 -- 172.21.15.40:0/182378553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 msgr2=0x7f655c149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 -- 172.21.15.40:0/182378553 >> v1:172.21.15.204:6789/0 conn(0x7f655c0b23d0 legacy=0x7f655c09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:53.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 -- 172.21.15.40:0/182378553 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0b08a0 con 0x7f655c073230 2024-04-02T16:11:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561a49640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c073230 0x7f655c0b27b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f654c009870 tx=0x7f654c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97642802836f3f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561248640 1 -- 172.21.15.40:0/182378553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a7070 con 0x7f655c073230 2024-04-02T16:11:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561248640 1 -- 172.21.15.40:0/182378553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c0b0070 con 0x7f655c073230 2024-04-02T16:11:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f6561248640 1 -- 172.21.15.40:0/182378553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ab890 con 0x7f655c073230 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 -- 172.21.15.40:0/182378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c073230 msgr2=0x7f655c0b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.804+0000 7f65644ae640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c073230 0x7f655c0b27b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f654c009870 tx=0x7f654c01c490 comp rx=0 tx=0).stop 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/182378553 shutdown_connections 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 --2- 172.21.15.40:0/182378553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c073230 0x7f655c0b27b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/182378553 >> 172.21.15.40:0/182378553 conn(0x7f655c0b8fd0 msgr2=0x7f655c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/182378553 shutdown_connections 2024-04-02T16:11:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/182378553 wait complete. 2024-04-02T16:11:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 Processor -- start 2024-04-02T16:11:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- start start 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c073230 0x7f655c144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c13e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f655c152810 con 0x7f655c0b23d0 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f655c152690 con 0x7f655c073230 2024-04-02T16:11:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f655c152990 con 0x7f655c148e50 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42960/0 (socket says 172.21.15.40:42960) 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 -- 172.21.15.40:0/1599823194 learned_addr learned my addr 172.21.15.40:0/1599823194 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6562a4b640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c13e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f656224a640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c073230 0x7f655c144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 -- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 msgr2=0x7f655c13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c13e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 -- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c073230 msgr2=0x7f655c144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c073230 0x7f655c144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 -- 172.21.15.40:0/1599823194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0b50e0 con 0x7f655c0b23d0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6561a49640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ad040 tx=0x7f654c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0be5f0 con 0x7f655c0b23d0 2024-04-02T16:11:53.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c0be7b0 con 0x7f655c0b23d0 2024-04-02T16:11:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0beba0 con 0x7f655c0b23d0 2024-04-02T16:11:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c13f6a0 con 0x7f655c0b23d0 2024-04-02T16:11:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c140080 con 0x7f655c0b23d0 2024-04-02T16:11:53.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.808+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c0685f0 con 0x7f655c0b23d0 2024-04-02T16:11:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.812+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f654c0b43f0 con 0x7f655c0b23d0 2024-04-02T16:11:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.812+0000 7f6552ffd640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 0x7f654006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.812+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f654c121780 con 0x7f655c0b23d0 2024-04-02T16:11:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.812+0000 7f656224a640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 0x7f654006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.812+0000 7f656224a640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 0x7f654006a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f655c13f070 tx=0x7f6558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:53.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.816+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f654c016110 con 0x7f655c0b23d0 2024-04-02T16:11:53.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.980+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f655c0b56e0 con 0x7f655c0b23d0 2024-04-02T16:11:53.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.980+0000 7f6552ffd640 1 -- 172.21.15.40:0/1599823194 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f654c0f1dd0 con 0x7f655c0b23d0 2024-04-02T16:11:53.982 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:53.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 msgr2=0x7f654006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 0x7f654006a840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f655c13f070 tx=0x7f6558009040 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 msgr2=0x7f655c140510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f654c0ad040 tx=0x7f654c01acd0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 shutdown_connections 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f655c148e50 0x7f655c13e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6540068380 0x7f654006a840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655c0b23d0 0x7f655c140510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 --2- 172.21.15.40:0/1599823194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f655c073230 0x7f655c144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 >> 172.21.15.40:0/1599823194 conn(0x7f655c0b8fd0 msgr2=0x7f655c06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 shutdown_connections 2024-04-02T16:11:53.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:53.984+0000 7f65644ae640 1 -- 172.21.15.40:0/1599823194 wait complete. 2024-04-02T16:11:54.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:54.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.124+0000 7ff334985640 1 Processor -- start 2024-04-02T16:11:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.124+0000 7ff334985640 1 -- start start 2024-04-02T16:11:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff334985640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff3300b23d0 0x7ff3300b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff334985640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3300b19e0 con 0x7ff330073230 2024-04-02T16:11:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff334985640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3300b05c0 con 0x7ff330148e50 2024-04-02T16:11:54.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff334985640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff3300b08a0 con 0x7ff3300b23d0 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32dd9b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff330073230 0x7ff33009daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60844/0 (socket says 172.21.15.40:60844) 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32dd9b640 1 -- 172.21.15.40:0/2110090649 learned_addr learned my addr 172.21.15.40:0/2110090649 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> v2:172.21.15.204:3300/0 conn(0x7ff3300b23d0 0x7ff3300b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258600557 0 0) 0x7ff3300b19e0 con 0x7ff330073230 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff31c003530 con 0x7ff330073230 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3433217520 0 0) 0x7ff31c003530 con 0x7ff330073230 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 >> v2:172.21.15.204:3300/0 conn(0x7ff3300b23d0 msgr2=0x7ff3300b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 --2- 172.21.15.40:0/2110090649 >> v2:172.21.15.204:3300/0 conn(0x7ff3300b23d0 0x7ff3300b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 >> v1:172.21.15.204:6789/0 conn(0x7ff330148e50 legacy=0x7ff330149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3300b0b80 con 0x7ff330073230 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7ff318001680 con 0x7ff330073230 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 >> v1:172.21.15.40:6789/0 conn(0x7ff330073230 legacy=0x7ff33009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff31c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff31c004810 0x7ff31c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff31c0051d0 0x7ff31c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3300b08a0 con 0x7ff31c0039c0 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c00c170 con 0x7ff31c0051d0 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c00c370 con 0x7ff31c004810 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> v2:172.21.15.204:3300/0 conn(0x7ff3300b23d0 0x7ff3300b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff31c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32ed9d640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff31c0051d0 0x7ff31c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32dd9b640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff31c004810 0x7ff31c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 -- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff31c004810 msgr2=0x7ff31c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff31c004810 0x7ff31c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 -- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff31c0051d0 msgr2=0x7ff31c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff31c0051d0 0x7ff31c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32e59c640 1 -- 172.21.15.40:0/2110090649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c003530 con 0x7ff31c0039c0 2024-04-02T16:11:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32ed9d640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff31c0051d0 0x7ff31c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:54.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.128+0000 7ff32ed9d640 1 -- 172.21.15.40:0/2110090649 reap_dead start 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff31c003de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff320009c90 tx=0x7ff32001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718b52bf9d9fa691 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320009860 con 0x7ff31c0039c0 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff320004020 con 0x7ff31c0039c0 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32d59a640 1 -- 172.21.15.40:0/2110090649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320004450 con 0x7ff31c0039c0 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 msgr2=0x7ff31c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff31c003de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff320009c90 tx=0x7ff32001cbf0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- 172.21.15.40:0/2110090649 shutdown_connections 2024-04-02T16:11:54.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 --2- 172.21.15.40:0/2110090649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff31c003de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- 172.21.15.40:0/2110090649 >> 172.21.15.40:0/2110090649 conn(0x7ff3300b8fd0 msgr2=0x7ff3300b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- 172.21.15.40:0/2110090649 shutdown_connections 2024-04-02T16:11:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- 172.21.15.40:0/2110090649 wait complete. 2024-04-02T16:11:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 Processor -- start 2024-04-02T16:11:54.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- start start 2024-04-02T16:11:54.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff330148620 0x7ff3301e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3301e2800 0x7ff3301e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301458e0 con 0x7ff31c0039c0 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff330145760 con 0x7ff3301e2800 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff334985640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff330145a60 con 0x7ff330148620 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42978/0 (socket says 172.21.15.40:42978) 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 -- 172.21.15.40:0/3791323700 learned_addr learned my addr 172.21.15.40:0/3791323700 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32dd9b640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff330148620 0x7ff3301e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 -- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff330148620 msgr2=0x7ff3301e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff330148620 0x7ff3301e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 -- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3301e2800 msgr2=0x7ff3301e2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3301e2800 0x7ff3301e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 -- 172.21.15.40:0/3791323700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3301d25c0 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff32e59c640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff32001ca20 tx=0x7ff31c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.132+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200ba020 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff320005db0 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3301d28a0 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200a58e0 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3301d2240 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3301d2c80 con 0x7ff31c0039c0 2024-04-02T16:11:54.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff3200bd020 con 0x7ff31c0039c0 2024-04-02T16:11:54.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff326ffd640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 0x7ff31006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff320120cb0 con 0x7ff31c0039c0 2024-04-02T16:11:54.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff32dd9b640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 0x7ff31006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.136+0000 7ff32dd9b640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 0x7ff31006a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff330073200 tx=0x7ff318009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.140+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff3301d2c80 con 0x7ff31c0039c0 2024-04-02T16:11:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.300+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3300685f0 con 0x7ff31c0039c0 2024-04-02T16:11:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.304+0000 7ff326ffd640 1 -- 172.21.15.40:0/3791323700 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7ff3200c4030 con 0x7ff31c0039c0 2024-04-02T16:11:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:54.306 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 msgr2=0x7ff31006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 0x7ff31006a840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff330073200 tx=0x7ff318009040 comp rx=0 tx=0).stop 2024-04-02T16:11:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 msgr2=0x7ff3300b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff32001ca20 tx=0x7ff31c005840 comp rx=0 tx=0).stop 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 shutdown_connections 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff330148620 0x7ff3301e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff310068380 0x7ff31006a840 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3301e2800 0x7ff3301e2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 --2- 172.21.15.40:0/3791323700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff31c0039c0 0x7ff3300b4620 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 >> 172.21.15.40:0/3791323700 conn(0x7ff3300b8fd0 msgr2=0x7ff33006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 shutdown_connections 2024-04-02T16:11:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.308+0000 7ff334985640 1 -- 172.21.15.40:0/3791323700 wait complete. 2024-04-02T16:11:54.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:54.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 Processor -- start 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 -- start start 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a000bb470 0x7f2a0006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a00141b50 0x7f2a0006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a000bb850 con 0x7f2a000b5860 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0006d0d0 con 0x7f2a000bb470 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a0752a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a000b6300 con 0x7f2a00141b50 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a000bb470 0x7f2a0006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2a000bb470 0x7f2a0006d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42994/0 (socket says 172.21.15.40:42994) 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 -- 172.21.15.40:0/516766289 learned_addr learned my addr 172.21.15.40:0/516766289 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 -- 172.21.15.40:0/516766289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a00141b50 msgr2=0x7f2a0006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a05ac7640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a00141b50 0x7f2a0006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a00141b50 0x7f2a0006dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 -- 172.21.15.40:0/516766289 >> v1:172.21.15.40:6789/0 conn(0x7f2a000b5860 legacy=0x7f2a00141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.440+0000 7f2a052c6640 1 -- 172.21.15.40:0/516766289 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a000b6b90 con 0x7f2a000bb470 2024-04-02T16:11:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a052c6640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a000bb470 0x7f2a0006d960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009870 tx=0x7f29fc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6172831dc433ab3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f29f7fff640 1 -- 172.21.15.40:0/516766289 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0b1070 con 0x7f2a000bb470 2024-04-02T16:11:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f29f7fff640 1 -- 172.21.15.40:0/516766289 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc0a8050 con 0x7f2a000bb470 2024-04-02T16:11:54.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f29f7fff640 1 -- 172.21.15.40:0/516766289 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0ac860 con 0x7f2a000bb470 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- 172.21.15.40:0/516766289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a000bb470 msgr2=0x7f2a0006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a000bb470 0x7f2a0006d960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009870 tx=0x7f29fc01c4d0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- 172.21.15.40:0/516766289 shutdown_connections 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a00141b50 0x7f2a0006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- 172.21.15.40:0/516766289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a000bb470 0x7f2a0006d960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- 172.21.15.40:0/516766289 >> 172.21.15.40:0/516766289 conn(0x7f2a0002c5d0 msgr2=0x7f2a0002c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- 172.21.15.40:0/516766289 shutdown_connections 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- 172.21.15.40:0/516766289 wait complete. 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 Processor -- start 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- start start 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000b5860 0x7f2a000b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000bb470 0x7f2a000b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2a001549e0 con 0x7f2a00141b50 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a00154860 con 0x7f2a000bb470 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a0752a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a00154b60 con 0x7f2a000b5860 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43002/0 (socket says 172.21.15.40:43002) 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 -- 172.21.15.40:0/3791907918 learned_addr learned my addr 172.21.15.40:0/3791907918 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a052c6640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000b5860 0x7f2a000b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a04ac5640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000bb470 0x7f2a000b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 -- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000b5860 msgr2=0x7f2a000b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000b5860 0x7f2a000b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 -- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000bb470 msgr2=0x7f2a000b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000bb470 0x7f2a000b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 -- 172.21.15.40:0/3791907918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a000b3590 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.444+0000 7f2a05ac7640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29f800cc30 tx=0x7f29f800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f800ce40 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f80039a0 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8003d50 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a000b5050 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a000b3800 con 0x7f2a00141b50 2024-04-02T16:11:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a00139bd0 con 0x7f2a00141b50 2024-04-02T16:11:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29f8003f70 con 0x7f2a00141b50 2024-04-02T16:11:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 0x7f29e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29f8086e40 con 0x7f2a00141b50 2024-04-02T16:11:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f2a052c6640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 0x7f29e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.448+0000 7f2a052c6640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 0x7f29e406a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0ae000 tx=0x7f29fc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.452+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2a00139bd0 con 0x7f2a00141b50 2024-04-02T16:11:54.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.608+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a00068d00 con 0x7f2a00141b50 2024-04-02T16:11:54.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.608+0000 7f29f5ffb640 1 -- 172.21.15.40:0/3791907918 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f29f8014030 con 0x7f2a00141b50 2024-04-02T16:11:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 msgr2=0x7f29e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 0x7f29e406a840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0ae000 tx=0x7f29fc0a7000 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 msgr2=0x7f2a001e0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29f800cc30 tx=0x7f29f800a5c0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 shutdown_connections 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29e4068380 0x7f29e406a840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00141b50 0x7f2a001e0260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a000bb470 0x7f2a000b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 --2- 172.21.15.40:0/3791907918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a000b5860 0x7f2a000b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 >> 172.21.15.40:0/3791907918 conn(0x7f2a0002c5d0 msgr2=0x7f2a000b78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 shutdown_connections 2024-04-02T16:11:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.612+0000 7f2a0752a640 1 -- 172.21.15.40:0/3791907918 wait complete. 2024-04-02T16:11:54.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:54.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.752+0000 7fd4875a8640 1 Processor -- start 2024-04-02T16:11:54.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.752+0000 7fd4875a8640 1 -- start start 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4800b5490 0x7fd4800aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd4800b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd48006d610 0x7fd48006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd48014b2d0 con 0x7fd48006d610 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800b5870 con 0x7fd4800b5490 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd4875a8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800b6070 con 0x7fd48014ae90 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd48006d610 0x7fd48006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd48006d610 0x7fd48006da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43004/0 (socket says 172.21.15.40:43004) 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 -- 172.21.15.40:0/4075475803 learned_addr learned my addr 172.21.15.40:0/4075475803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485344640 1 --2- 172.21.15.40:0/4075475803 >> v2:172.21.15.204:3300/0 conn(0x7fd4800b5490 0x7fd4800aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd484b43640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd4800b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 -- 172.21.15.40:0/4075475803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 msgr2=0x7fd4800b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd4800b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 -- 172.21.15.40:0/4075475803 >> v2:172.21.15.204:3300/0 conn(0x7fd4800b5490 msgr2=0x7fd4800aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 --2- 172.21.15.40:0/4075475803 >> v2:172.21.15.204:3300/0 conn(0x7fd4800b5490 0x7fd4800aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 -- 172.21.15.40:0/4075475803 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4800aef60 con 0x7fd48006d610 2024-04-02T16:11:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd485b45640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd48006da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd48009f330 tx=0x7fd47801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155b0573c01c12b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd477fff640 1 -- 172.21.15.40:0/4075475803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780a5700 con 0x7fd48006d610 2024-04-02T16:11:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd477fff640 1 -- 172.21.15.40:0/4075475803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4780a58a0 con 0x7fd48006d610 2024-04-02T16:11:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.756+0000 7fd477fff640 1 -- 172.21.15.40:0/4075475803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780a5c70 con 0x7fd48006d610 2024-04-02T16:11:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- 172.21.15.40:0/4075475803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 msgr2=0x7fd48006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd48006da80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd48009f330 tx=0x7fd47801c8a0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- 172.21.15.40:0/4075475803 shutdown_connections 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd48006da80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- 172.21.15.40:0/4075475803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd4800b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- 172.21.15.40:0/4075475803 >> v2:172.21.15.204:3300/0 conn(0x7fd4800b5490 0x7fd4800aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- 172.21.15.40:0/4075475803 >> 172.21.15.40:0/4075475803 conn(0x7fd48002cdd0 msgr2=0x7fd4800b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- 172.21.15.40:0/4075475803 shutdown_connections 2024-04-02T16:11:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- 172.21.15.40:0/4075475803 wait complete. 2024-04-02T16:11:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 Processor -- start 2024-04-02T16:11:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- start start 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4800b5490 0x7fd4801468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd48014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800b6310 con 0x7fd48006d610 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800733b0 con 0x7fd4800b5490 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd4875a8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800b6490 con 0x7fd48014ae90 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43010/0 (socket says 172.21.15.40:43010) 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 -- 172.21.15.40:0/1958973717 learned_addr learned my addr 172.21.15.40:0/1958973717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd484b43640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4800b5490 0x7fd4801468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485b45640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd48014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 -- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 msgr2=0x7fd48014d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd48014d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 -- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4800b5490 msgr2=0x7fd4801468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4800b5490 0x7fd4801468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 -- 172.21.15.40:0/1958973717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48014e6f0 con 0x7fd48006d610 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.760+0000 7fd485344640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00d800 tx=0x7fd47c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00b6c0 con 0x7fd48006d610 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c019070 con 0x7fd48006d610 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48014e9d0 con 0x7fd48006d610 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00ba60 con 0x7fd48006d610 2024-04-02T16:11:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4800b2f30 con 0x7fd48006d610 2024-04-02T16:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48014c7f0 con 0x7fd48006d610 2024-04-02T16:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd47c00bc00 con 0x7fd48006d610 2024-04-02T16:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 0x7fd46406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd47c086dd0 con 0x7fd48006d610 2024-04-02T16:11:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd484b43640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 0x7fd46406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.764+0000 7fd484b43640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 0x7fd46406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4800b7470 tx=0x7fd470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.768+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd47c057420 con 0x7fd48006d610 2024-04-02T16:11:54.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.932+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4800685f0 con 0x7fd48006d610 2024-04-02T16:11:54.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd475ffb640 1 -- 172.21.15.40:0/1958973717 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd47c088020 con 0x7fd48006d610 2024-04-02T16:11:54.936 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:54.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 msgr2=0x7fd46406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 0x7fd46406a840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4800b7470 tx=0x7fd470008040 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 msgr2=0x7fd4801e2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00d800 tx=0x7fd47c00dcc0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 shutdown_connections 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd48014ae90 0x7fd48014d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd464068380 0x7fd46406a840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4800b5490 0x7fd4801468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 --2- 172.21.15.40:0/1958973717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd48006d610 0x7fd4801e2f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 >> 172.21.15.40:0/1958973717 conn(0x7fd48002cdd0 msgr2=0x7fd4800b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 shutdown_connections 2024-04-02T16:11:54.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:54.936+0000 7fd4875a8640 1 -- 172.21.15.40:0/1958973717 wait complete. 2024-04-02T16:11:54.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 Processor -- start 2024-04-02T16:11:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 -- start start 2024-04-02T16:11:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd9c0bd990 0x7efd9c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efd9c0b80b0 0x7efd9c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9c14b8f0 con 0x7efd9c14b480 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c0b7040 con 0x7efd9c0bd990 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda36c8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efd9c0b5000 con 0x7efd9c0b80b0 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd9c0bd990 0x7efd9c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd9c0bd990 0x7efd9c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43022/0 (socket says 172.21.15.40:43022) 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda0c63640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efd9c14b480 0x7efd9c0b79a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60872/0 (socket says 172.21.15.40:60872) 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 -- 172.21.15.40:0/1388630471 learned_addr learned my addr 172.21.15.40:0/1388630471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1c65640 1 --2- 172.21.15.40:0/1388630471 >> v2:172.21.15.204:3300/0 conn(0x7efd9c0b80b0 0x7efd9c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 -- 172.21.15.40:0/1388630471 >> v2:172.21.15.204:3300/0 conn(0x7efd9c0b80b0 msgr2=0x7efd9c1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 --2- 172.21.15.40:0/1388630471 >> v2:172.21.15.204:3300/0 conn(0x7efd9c0b80b0 0x7efd9c1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 -- 172.21.15.40:0/1388630471 >> v1:172.21.15.40:6789/0 conn(0x7efd9c14b480 legacy=0x7efd9c0b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efda1464640 1 -- 172.21.15.40:0/1388630471 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c0b52e0 con 0x7efd9c0bd990 2024-04-02T16:11:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.080+0000 7efd93fff640 1 -- 172.21.15.40:0/1388630471 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819056366 0 0) 0x7efd9c14b8f0 con 0x7efd9c14b480 2024-04-02T16:11:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- 172.21.15.40:0/1388630471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0bd990 0x7efd9c14af40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0b55c0 tx=0x7efd8c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41fd04158c783fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd93fff640 1 -- 172.21.15.40:0/1388630471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b0070 con 0x7efd9c0bd990 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd93fff640 1 -- 172.21.15.40:0/1388630471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c0ab600 con 0x7efd9c0bd990 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd93fff640 1 -- 172.21.15.40:0/1388630471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0ab9d0 con 0x7efd9c0bd990 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- 172.21.15.40:0/1388630471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0bd990 msgr2=0x7efd9c14af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- 172.21.15.40:0/1388630471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0bd990 0x7efd9c14af40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0b55c0 tx=0x7efd8c01c490 comp rx=0 tx=0).stop 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- 172.21.15.40:0/1388630471 shutdown_connections 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- 172.21.15.40:0/1388630471 >> v2:172.21.15.204:3300/0 conn(0x7efd9c0b80b0 0x7efd9c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- 172.21.15.40:0/1388630471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0bd990 0x7efd9c14af40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- 172.21.15.40:0/1388630471 >> 172.21.15.40:0/1388630471 conn(0x7efd9c02ca50 msgr2=0x7efd9c0bd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- 172.21.15.40:0/1388630471 shutdown_connections 2024-04-02T16:11:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- 172.21.15.40:0/1388630471 wait complete. 2024-04-02T16:11:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 Processor -- start 2024-04-02T16:11:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- start start 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 0x7efd9c1e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9c14b480 0x7efd9c1e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0b6100 con 0x7efd9c0b80b0 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c0b6280 con 0x7efd9c0bd990 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda36c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c155100 con 0x7efd9c14b480 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43026/0 (socket says 172.21.15.40:43026) 2024-04-02T16:11:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 -- 172.21.15.40:0/296647379 learned_addr learned my addr 172.21.15.40:0/296647379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda0c63640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 0x7efd9c1e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1c65640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9c14b480 0x7efd9c1e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 -- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9c14b480 msgr2=0x7efd9c1e0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9c14b480 0x7efd9c1e0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 -- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 msgr2=0x7efd9c1e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 0x7efd9c1e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 -- 172.21.15.40:0/296647379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c143370 con 0x7efd9c0b80b0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda0c63640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 0x7efd9c1e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efda1464640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0ae040 tx=0x7efd8c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0ba020 con 0x7efd9c0b80b0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c0b6620 con 0x7efd9c0b80b0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.084+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b0040 con 0x7efd9c0b80b0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c143090 con 0x7efd9c0b80b0 2024-04-02T16:11:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c0bcaa0 con 0x7efd9c0b80b0 2024-04-02T16:11:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c068d00 con 0x7efd9c0b80b0 2024-04-02T16:11:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7efd8c0bd020 con 0x7efd9c0b80b0 2024-04-02T16:11:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efd91ffb640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 0x7efd8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd8c120c30 con 0x7efd9c0b80b0 2024-04-02T16:11:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efda0c63640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 0x7efd8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.088+0000 7efda0c63640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 0x7efd8006a870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd98007fd0 tx=0x7efd98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.092+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7efd8c0c4030 con 0x7efd9c0b80b0 2024-04-02T16:11:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.260+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd9c0b67a0 con 0x7efd9c0b80b0 2024-04-02T16:11:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.260+0000 7efd91ffb640 1 -- 172.21.15.40:0/296647379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7efd9c143370 con 0x7efd9c0b80b0 2024-04-02T16:11:55.264 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 msgr2=0x7efd8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 0x7efd8006a870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd98007fd0 tx=0x7efd98009040 comp rx=0 tx=0).stop 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 msgr2=0x7efd9c14a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0ae040 tx=0x7efd8c01acd0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 shutdown_connections 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd9c14b480 0x7efd9c1e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7efd800683b0 0x7efd8006a870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd9c0bd990 0x7efd9c1e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 --2- 172.21.15.40:0/296647379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd9c0b80b0 0x7efd9c14a770 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 >> 172.21.15.40:0/296647379 conn(0x7efd9c02ca50 msgr2=0x7efd9c1416e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 shutdown_connections 2024-04-02T16:11:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.264+0000 7efda36c8640 1 -- 172.21.15.40:0/296647379 wait complete. 2024-04-02T16:11:55.333 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-04-02T16:11:55.333 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-02T16:11:55.334 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.460+0000 7f95e57ee640 1 Processor -- start 2024-04-02T16:11:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.460+0000 7f95e57ee640 1 -- start start 2024-04-02T16:11:55.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95e57ee640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 0x7f95e01455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95e57ee640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e00b3a30 con 0x7f95e0145b00 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95e57ee640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e00b4230 con 0x7f95e0072e50 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95e57ee640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95e00b2360 con 0x7f95e0072a70 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95e0145b00 0x7f95e00718d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40508/0 (socket says 172.21.15.40:40508) 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 -- 172.21.15.40:0/4122161259 learned_addr learned my addr 172.21.15.40:0/4122161259 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95deffd640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 0x7f95e01455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239340558 0 0) 0x7f95e00b3a30 con 0x7f95e0145b00 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95cc003530 con 0x7f95e0145b00 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 927422654 0 0) 0x7f95cc003530 con 0x7f95e0145b00 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 msgr2=0x7f95e01455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 0x7f95e01455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 >> v1:172.21.15.204:6789/0 conn(0x7f95e0072e50 legacy=0x7f95e014fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e00b1fb0 con 0x7f95e0145b00 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f95c8001680 con 0x7f95e0145b00 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 >> v1:172.21.15.40:6789/0 conn(0x7f95e0145b00 legacy=0x7f95e00718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039f0 0x7f95cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004810 0x7f95cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 0x7f95cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95e00b2360 con 0x7f95cc004810 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc00bab0 con 0x7f95cc0039f0 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc00bcb0 con 0x7f95e0072a70 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004810 0x7f95cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95deffd640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039f0 0x7f95cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 -- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 msgr2=0x7f95cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0072a70 0x7f95cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 -- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039f0 msgr2=0x7f95cc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039f0 0x7f95cc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 -- 172.21.15.40:0/4122161259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc003530 con 0x7f95cc004810 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95d7fff640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004810 0x7f95cc004c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95e0145f00 tx=0x7f95c801f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9c1bfadb754e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8007c70 con 0x7f95cc004810 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c80ac070 con 0x7f95cc004810 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.464+0000 7f95de7fc640 1 -- 172.21.15.40:0/4122161259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8005b20 con 0x7f95cc004810 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4122161259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004810 msgr2=0x7f95cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4122161259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004810 0x7f95cc004c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f95e0145f00 tx=0x7f95c801f2b0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 -- 172.21.15.40:0/4122161259 reap_dead start 2024-04-02T16:11:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4122161259 shutdown_connections 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4122161259 >> 172.21.15.40:0/4122161259 conn(0x7f95e00b7d10 msgr2=0x7f95e00b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4122161259 shutdown_connections 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4122161259 wait complete. 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 Processor -- start 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- start start 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc004810 0x7f95e0149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0142810 0x7f95e0142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95e00b3490 con 0x7f95e01e0510 2024-04-02T16:11:55.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95e00b3da0 con 0x7f95cc004810 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95e01455c0 con 0x7f95e0142810 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57110/0 (socket says 172.21.15.40:57110) 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 -- 172.21.15.40:0/4155209635 learned_addr learned my addr 172.21.15.40:0/4155209635 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95d7fff640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0142810 0x7f95e0142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95deffd640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc004810 0x7f95e0149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 -- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0142810 msgr2=0x7f95e0142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0142810 0x7f95e0142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 -- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc004810 msgr2=0x7f95e0149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc004810 0x7f95e0149380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 -- 172.21.15.40:0/4155209635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e0142110 con 0x7f95e01e0510 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95df7fe640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95e00706a0 tx=0x7f95d800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8017070 con 0x7f95e01e0510 2024-04-02T16:11:55.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8005d50 con 0x7f95e01e0510 2024-04-02T16:11:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800a850 con 0x7f95e01e0510 2024-04-02T16:11:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e0140df0 con 0x7f95e01e0510 2024-04-02T16:11:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.468+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e01407f0 con 0x7f95e01e0510 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e01423e0 con 0x7f95e01e0510 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95d801a070 con 0x7f95e01e0510 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95d77fe640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 0x7f95c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f95d8085ca0 con 0x7f95e01e0510 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95deffd640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 0x7f95c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.472+0000 7f95deffd640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 0x7f95c006a870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009a50 tx=0x7f95d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.476+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f95e01423e0 con 0x7f95e01e0510 2024-04-02T16:11:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.636+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95e0068d00 con 0x7f95e01e0510 2024-04-02T16:11:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.640+0000 7f95d77fe640 1 -- 172.21.15.40:0/4155209635 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f95d8056260 con 0x7f95e01e0510 2024-04-02T16:11:55.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:55.642 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 msgr2=0x7f95c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 0x7f95c006a870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009a50 tx=0x7f95d0008040 comp rx=0 tx=0).stop 2024-04-02T16:11:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 msgr2=0x7f95e01e0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95e00706a0 tx=0x7f95d800cb90 comp rx=0 tx=0).stop 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 shutdown_connections 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e0142810 0x7f95e0142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95c00683b0 0x7f95c006a870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e01e0510 0x7f95e01e0990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 --2- 172.21.15.40:0/4155209635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc004810 0x7f95e0149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 >> 172.21.15.40:0/4155209635 conn(0x7f95e00b7d10 msgr2=0x7f95e009fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 shutdown_connections 2024-04-02T16:11:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.644+0000 7f95e57ee640 1 -- 172.21.15.40:0/4155209635 wait complete. 2024-04-02T16:11:55.665 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:55.666 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:55.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 Processor -- start 2024-04-02T16:11:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 -- start start 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb040b23d0 0x7feb040b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb04073230 0x7feb0409daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb0409dfe0 0x7feb04148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7feb040b19e0 con 0x7feb040b23d0 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feb040b05c0 con 0x7feb04073230 2024-04-02T16:11:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb08f15640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb040b08a0 con 0x7feb0409dfe0 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb040b23d0 0x7feb040b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb040b23d0 0x7feb040b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57118/0 (socket says 172.21.15.40:57118) 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 -- 172.21.15.40:0/4021972069 learned_addr learned my addr 172.21.15.40:0/4021972069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb02d9d640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb0409dfe0 0x7feb04148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb01d9b640 1 --2- 172.21.15.40:0/4021972069 >> v2:172.21.15.204:3300/0 conn(0x7feb04073230 0x7feb0409daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 -- 172.21.15.40:0/4021972069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb0409dfe0 msgr2=0x7feb04148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb0409dfe0 0x7feb04148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 -- 172.21.15.40:0/4021972069 >> v2:172.21.15.204:3300/0 conn(0x7feb04073230 msgr2=0x7feb0409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 --2- 172.21.15.40:0/4021972069 >> v2:172.21.15.204:3300/0 conn(0x7feb04073230 0x7feb0409daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.788+0000 7feb0259c640 1 -- 172.21.15.40:0/4021972069 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb040b0b80 con 0x7feb040b23d0 2024-04-02T16:11:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb0259c640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb040b27b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb04148b00 tx=0x7feafc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a965509ecf8d2443 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb0159a640 1 -- 172.21.15.40:0/4021972069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0b0070 con 0x7feb040b23d0 2024-04-02T16:11:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb0159a640 1 -- 172.21.15.40:0/4021972069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feafc0ab600 con 0x7feb040b23d0 2024-04-02T16:11:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb0159a640 1 -- 172.21.15.40:0/4021972069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0ab9d0 con 0x7feb040b23d0 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 -- 172.21.15.40:0/4021972069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 msgr2=0x7feb040b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb040b27b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb04148b00 tx=0x7feafc01c490 comp rx=0 tx=0).stop 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 -- 172.21.15.40:0/4021972069 shutdown_connections 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb0409dfe0 0x7feb04148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 --2- 172.21.15.40:0/4021972069 >> v2:172.21.15.204:3300/0 conn(0x7feb04073230 0x7feb0409daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 --2- 172.21.15.40:0/4021972069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb040b27b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 -- 172.21.15.40:0/4021972069 >> 172.21.15.40:0/4021972069 conn(0x7feb040b8fd0 msgr2=0x7feb040b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 -- 172.21.15.40:0/4021972069 shutdown_connections 2024-04-02T16:11:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.792+0000 7feb08f15640 1 -- 172.21.15.40:0/4021972069 wait complete. 2024-04-02T16:11:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 Processor -- start 2024-04-02T16:11:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- start start 2024-04-02T16:11:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 0x7feb04071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 0x7feb04072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb041460f0 con 0x7feb040b23d0 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb04145f70 con 0x7feb0409dfe0 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb04146270 con 0x7feb04073230 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57126/0 (socket says 172.21.15.40:57126) 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 -- 172.21.15.40:0/40732397 learned_addr learned my addr 172.21.15.40:0/40732397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb0259c640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 0x7feb04071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb01d9b640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 0x7feb04072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 -- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 msgr2=0x7feb04071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 0x7feb04071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 -- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 msgr2=0x7feb04072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 0x7feb04072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 -- 172.21.15.40:0/40732397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb04072740 con 0x7feb040b23d0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb0259c640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 0x7feb04071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb01d9b640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 0x7feb04072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb02d9d640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feaf4007e60 tx=0x7feaf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4014420 con 0x7feb040b23d0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf4019070 con 0x7feb040b23d0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb04071020 con 0x7feb040b23d0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4014820 con 0x7feb040b23d0 2024-04-02T16:11:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb04071450 con 0x7feb040b23d0 2024-04-02T16:11:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.796+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb040685f0 con 0x7feb040b23d0 2024-04-02T16:11:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.800+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feaf40149c0 con 0x7feb040b23d0 2024-04-02T16:11:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.800+0000 7feafaffd640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 0x7feae406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.800+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feaf40870c0 con 0x7feb040b23d0 2024-04-02T16:11:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.800+0000 7feb0259c640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 0x7feae406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.800+0000 7feb0259c640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 0x7feae406a870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feafc0ad000 tx=0x7feafc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.804+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7feaf4057710 con 0x7feb040b23d0 2024-04-02T16:11:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.968+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb0409ddc0 con 0x7feb040b23d0 2024-04-02T16:11:55.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.968+0000 7feafaffd640 1 -- 172.21.15.40:0/40732397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7feaf4057010 con 0x7feb040b23d0 2024-04-02T16:11:55.970 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:55.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 msgr2=0x7feae406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 0x7feae406a870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feafc0ad000 tx=0x7feafc008040 comp rx=0 tx=0).stop 2024-04-02T16:11:55.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 msgr2=0x7feb0413bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:55.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feaf4007e60 tx=0x7feaf400e4b0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 shutdown_connections 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feae40683b0 0x7feae406a870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb040b23d0 0x7feb0413bee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb0409dfe0 0x7feb04072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 --2- 172.21.15.40:0/40732397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb04073230 0x7feb04071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 >> 172.21.15.40:0/40732397 conn(0x7feb040b8fd0 msgr2=0x7feb0413d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 shutdown_connections 2024-04-02T16:11:55.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:55.972+0000 7feb08f15640 1 -- 172.21.15.40:0/40732397 wait complete. 2024-04-02T16:11:55.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.116+0000 7f63c79cc640 1 Processor -- start 2024-04-02T16:11:56.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.116+0000 7f63c79cc640 1 -- start start 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c79cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63c00b23d0 0x7f63c00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c79cc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c0073230 0x7f63c009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c79cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c00b19e0 con 0x7f63c009dfe0 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c79cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f63c00b05c0 con 0x7f63c00b23d0 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c79cc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63c00b08a0 con 0x7f63c0073230 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63c00b23d0 0x7f63c00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63c00b23d0 0x7f63c00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57138/0 (socket says 172.21.15.40:57138) 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5f69640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63c009dfe0 0x7f63c0148e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40516/0 (socket says 172.21.15.40:40516) 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 -- 172.21.15.40:0/1913696415 learned_addr learned my addr 172.21.15.40:0/1913696415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c4f67640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c0073230 0x7f63c009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 -- 172.21.15.40:0/1913696415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c0073230 msgr2=0x7f63c009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:56.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c0073230 0x7f63c009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 -- 172.21.15.40:0/1913696415 >> v1:172.21.15.40:6789/0 conn(0x7f63c009dfe0 legacy=0x7f63c0148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63b3fff640 1 -- 172.21.15.40:0/1913696415 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888088492 0 0) 0x7f63c00b19e0 con 0x7f63c009dfe0 2024-04-02T16:11:56.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 -- 172.21.15.40:0/1913696415 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c00b0b80 con 0x7f63c00b23d0 2024-04-02T16:11:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63c5768640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c00b27b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009870 tx=0x7f63b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f744fa5fc91c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63b3fff640 1 -- 172.21.15.40:0/1913696415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40ab5f0 con 0x7f63c00b23d0 2024-04-02T16:11:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63b3fff640 1 -- 172.21.15.40:0/1913696415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b40b0070 con 0x7f63c00b23d0 2024-04-02T16:11:56.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.120+0000 7f63b3fff640 1 -- 172.21.15.40:0/1913696415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40ab9b0 con 0x7f63c00b23d0 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/1913696415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 msgr2=0x7f63c00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c00b27b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009870 tx=0x7f63b401c4d0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/1913696415 shutdown_connections 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c0073230 0x7f63c009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- 172.21.15.40:0/1913696415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c00b27b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/1913696415 >> 172.21.15.40:0/1913696415 conn(0x7f63c00b8fd0 msgr2=0x7f63c00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/1913696415 shutdown_connections 2024-04-02T16:11:56.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/1913696415 wait complete. 2024-04-02T16:11:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 Processor -- start 2024-04-02T16:11:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- start start 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 0x7f63c009d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c009dfe0 0x7f63c00af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0152a20 con 0x7f63c00b23d0 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01528a0 con 0x7f63c0073230 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63c0152ba0 con 0x7f63c009dfe0 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57146/0 (socket says 172.21.15.40:57146) 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 -- 172.21.15.40:0/3663441939 learned_addr learned my addr 172.21.15.40:0/3663441939 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:56.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c4f67640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c009dfe0 0x7f63c00af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5768640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 0x7f63c009d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 -- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c009dfe0 msgr2=0x7f63c00af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c009dfe0 0x7f63c00af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 -- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 msgr2=0x7f63c009d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 0x7f63c009d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 -- 172.21.15.40:0/3663441939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c01466e0 con 0x7f63c00b23d0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5768640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 0x7f63c009d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c5f69640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001570 tx=0x7f63b800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8018070 con 0x7f63c00b23d0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b800a640 con 0x7f63c00b23d0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c009d910 con 0x7f63c00b23d0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b800aa30 con 0x7f63c00b23d0 2024-04-02T16:11:56.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.124+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c00ae0b0 con 0x7f63c00b23d0 2024-04-02T16:11:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c00aecd0 con 0x7f63c00b23d0 2024-04-02T16:11:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f63b801b070 con 0x7f63c00b23d0 2024-04-02T16:11:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63b1ffb640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 0x7f63a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f63b8085ea0 con 0x7f63c00b23d0 2024-04-02T16:11:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63c5768640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 0x7f63a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.128+0000 7f63c5768640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 0x7f63a406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63b40af000 tx=0x7f63b40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.132+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f63c01466e0 con 0x7f63c00b23d0 2024-04-02T16:11:56.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.296+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f63c00685f0 con 0x7f63c00b23d0 2024-04-02T16:11:56.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.296+0000 7f63b1ffb640 1 -- 172.21.15.40:0/3663441939 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f63b8055ea0 con 0x7f63c00b23d0 2024-04-02T16:11:56.300 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:56.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 msgr2=0x7f63a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 0x7f63a406a840 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63b40af000 tx=0x7f63b40a7000 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 msgr2=0x7f63c009b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63b8001570 tx=0x7f63b800cca0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 shutdown_connections 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c009dfe0 0x7f63c00af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f63a4068380 0x7f63a406a840 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c00b23d0 0x7f63c009b0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 --2- 172.21.15.40:0/3663441939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63c0073230 0x7f63c009d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 >> 172.21.15.40:0/3663441939 conn(0x7f63c00b8fd0 msgr2=0x7f63c006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 shutdown_connections 2024-04-02T16:11:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.300+0000 7f63c79cc640 1 -- 172.21.15.40:0/3663441939 wait complete. 2024-04-02T16:11:56.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 Processor -- start 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 -- start start 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f371006c8a0 0x7f37100af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37100b43a0 0x7f371014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37100ae830 con 0x7f37100b3800 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37100b2b50 con 0x7f371006c8a0 2024-04-02T16:11:56.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f371515b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37100b0300 con 0x7f37100b43a0 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f371006c8a0 0x7f37100af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37100b3800 0x7f37100b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40520/0 (socket says 172.21.15.40:40520) 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370659c640 1 -- 172.21.15.40:0/3283599022 learned_addr learned my addr 172.21.15.40:0/3283599022 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370f59e640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37100b43a0 0x7f371014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370e59c640 1 -- 172.21.15.40:0/3283599022 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3583885523 0 0) 0x7f37100ae830 con 0x7f37100b3800 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370e59c640 1 -- 172.21.15.40:0/3283599022 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36f0003530 con 0x7f37100b3800 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370ed9d640 1 -- 172.21.15.40:0/3283599022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37100b43a0 msgr2=0x7f371014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370ed9d640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37100b43a0 0x7f371014f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370ed9d640 1 -- 172.21.15.40:0/3283599022 >> v1:172.21.15.40:6789/0 conn(0x7f37100b3800 legacy=0x7f37100b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.432+0000 7f370ed9d640 1 -- 172.21.15.40:0/3283599022 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371006cc80 con 0x7f371006c8a0 2024-04-02T16:11:56.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f370ed9d640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371006c8a0 0x7f37100af150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097c0 tx=0x7f36f801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b00ecf8c78a5584 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f370e59c640 1 -- 172.21.15.40:0/3283599022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80b0070 con 0x7f371006c8a0 2024-04-02T16:11:56.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f370e59c640 1 -- 172.21.15.40:0/3283599022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f80ab600 con 0x7f371006c8a0 2024-04-02T16:11:56.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f370e59c640 1 -- 172.21.15.40:0/3283599022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80ab9d0 con 0x7f371006c8a0 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- 172.21.15.40:0/3283599022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371006c8a0 msgr2=0x7f37100af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371006c8a0 0x7f37100af150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097c0 tx=0x7f36f801c380 comp rx=0 tx=0).stop 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- 172.21.15.40:0/3283599022 shutdown_connections 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37100b43a0 0x7f371014f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 --2- 172.21.15.40:0/3283599022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f371006c8a0 0x7f37100af150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- 172.21.15.40:0/3283599022 >> 172.21.15.40:0/3283599022 conn(0x7f37100b8fd0 msgr2=0x7f37100b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- 172.21.15.40:0/3283599022 shutdown_connections 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- 172.21.15.40:0/3283599022 wait complete. 2024-04-02T16:11:56.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 Processor -- start 2024-04-02T16:11:56.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.436+0000 7f371515b640 1 -- start start 2024-04-02T16:11:56.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f371006c8a0 0x7f371014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37100b43a0 0x7f37101dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37100b3130 con 0x7f37100b3800 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37100af690 con 0x7f37100b43a0 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3710071590 con 0x7f371006c8a0 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57158/0 (socket says 172.21.15.40:57158) 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 -- 172.21.15.40:0/3004612003 learned_addr learned my addr 172.21.15.40:0/3004612003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 -- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f371006c8a0 msgr2=0x7f371014dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370ed9d640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f371006c8a0 0x7f371014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f371006c8a0 0x7f371014dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370f59e640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37100b43a0 0x7f37101dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 -- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37100b43a0 msgr2=0x7f37101dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37100b43a0 0x7f37101dfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 -- 172.21.15.40:0/3004612003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371014b580 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370659c640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f370000c460 tx=0x7f370000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700017070 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3700005d10 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3710070f40 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37000072b0 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371014e680 con 0x7f37100b3800 2024-04-02T16:11:56.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37100685f0 con 0x7f37100b3800 2024-04-02T16:11:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f370001a070 con 0x7f37100b3800 2024-04-02T16:11:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 0x7f36f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3700085c70 con 0x7f37100b3800 2024-04-02T16:11:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.440+0000 7f370ed9d640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 0x7f36f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.444+0000 7f370ed9d640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 0x7f36f406a870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36f80ad000 tx=0x7f36f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.444+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f37000562c0 con 0x7f37100b3800 2024-04-02T16:11:56.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.604+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37100b32b0 con 0x7f37100b3800 2024-04-02T16:11:56.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f3707fff640 1 -- 172.21.15.40:0/3004612003 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3700055bc0 con 0x7f37100b3800 2024-04-02T16:11:56.613 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:56.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 msgr2=0x7f36f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 0x7f36f406a870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f36f80ad000 tx=0x7f36f8008040 comp rx=0 tx=0).stop 2024-04-02T16:11:56.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 msgr2=0x7f371014ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f370000c460 tx=0x7f370000c920 comp rx=0 tx=0).stop 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 shutdown_connections 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f36f40683b0 0x7f36f406a870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37100b43a0 0x7f37101dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37100b3800 0x7f371014ab50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 --2- 172.21.15.40:0/3004612003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f371006c8a0 0x7f371014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 >> 172.21.15.40:0/3004612003 conn(0x7f37100b8fd0 msgr2=0x7f37100672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 shutdown_connections 2024-04-02T16:11:56.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.608+0000 7f371515b640 1 -- 172.21.15.40:0/3004612003 wait complete. 2024-04-02T16:11:56.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-02T16:11:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 Processor -- start 2024-04-02T16:11:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 -- start start 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43d80ae080 0x7f43d80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d80b3210 con 0x7f43d80b1c10 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d806db10 con 0x7f43d80b2780 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43e0409640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f43d80ae460 con 0x7f43d80ae080 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43dd9a4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43d80b1c10 0x7f43d80b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40532/0 (socket says 172.21.15.40:40532) 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43dd9a4640 1 -- 172.21.15.40:0/2965440883 learned_addr learned my addr 172.21.15.40:0/2965440883 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43de1a5640 1 --2- 172.21.15.40:0/2965440883 >> v2:172.21.15.40:3300/0 conn(0x7f43d80ae080 0x7f43d80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43dd1a3640 1 -- 172.21.15.40:0/2965440883 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3340698810 0 0) 0x7f43d80b3210 con 0x7f43d80b1c10 2024-04-02T16:11:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43dd1a3640 1 -- 172.21.15.40:0/2965440883 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43c4003530 con 0x7f43d80b1c10 2024-04-02T16:11:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43de1a5640 1 -- 172.21.15.40:0/2965440883 >> v1:172.21.15.204:6789/0 conn(0x7f43d80b2780 legacy=0x7f43d81522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43de1a5640 1 -- 172.21.15.40:0/2965440883 >> v1:172.21.15.40:6789/0 conn(0x7f43d80b1c10 legacy=0x7f43d80b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.752+0000 7f43de1a5640 1 -- 172.21.15.40:0/2965440883 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d80af670 con 0x7f43d80ae080 2024-04-02T16:11:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- 172.21.15.40:0/2965440883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d80b0300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43c80097c0 tx=0x7f43c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c58f4bdf6f4ba1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43dd1a3640 1 -- 172.21.15.40:0/2965440883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b0070 con 0x7f43d80ae080 2024-04-02T16:11:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43dd1a3640 1 -- 172.21.15.40:0/2965440883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80ab600 con 0x7f43d80ae080 2024-04-02T16:11:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43dd1a3640 1 -- 172.21.15.40:0/2965440883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80ab9b0 con 0x7f43d80ae080 2024-04-02T16:11:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/2965440883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 msgr2=0x7f43d80b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 --2- 172.21.15.40:0/2965440883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d80b0300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43c80097c0 tx=0x7f43c801c460 comp rx=0 tx=0).stop 2024-04-02T16:11:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/2965440883 shutdown_connections 2024-04-02T16:11:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 --2- 172.21.15.40:0/2965440883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d80b0300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/2965440883 >> 172.21.15.40:0/2965440883 conn(0x7f43d80b8fd0 msgr2=0x7f43d80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/2965440883 shutdown_connections 2024-04-02T16:11:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/2965440883 wait complete. 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 Processor -- start 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- start start 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d80b1c10 0x7f43d8071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 0x7f43d806f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80aeeb0 con 0x7f43d80ae080 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43d80af2f0 con 0x7f43d80b1c10 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f43d80af030 con 0x7f43d80b2780 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57180/0 (socket says 172.21.15.40:57180) 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 -- 172.21.15.40:0/1631773350 learned_addr learned my addr 172.21.15.40:0/1631773350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de9a6640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 0x7f43d806f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 -- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 msgr2=0x7f43d806f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43dd9a4640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d80b1c10 0x7f43d8071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 0x7f43d806f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 -- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d80b1c10 msgr2=0x7f43d8071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d80b1c10 0x7f43d8071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 -- 172.21.15.40:0/1631773350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d809aa70 con 0x7f43d80ae080 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de9a6640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 0x7f43d806f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43de1a5640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43c80ae040 tx=0x7f43c8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b9020 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80ab9b0 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d809c860 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80b0040 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.756+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d809c3d0 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8146ef0 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f43c80bc020 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43ceffd640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 0x7f43bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43c8120920 con 0x7f43d80ae080 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43dd9a4640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 0x7f43bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:56.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.760+0000 7f43dd9a4640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 0x7f43bc06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43d4007fd0 tx=0x7f43d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:56.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.764+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f43c80c3030 con 0x7f43d80ae080 2024-04-02T16:11:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:56.980+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f43d80b7a50 con 0x7f43d80ae080 2024-04-02T16:11:57.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.948+0000 7f43ceffd640 1 -- 172.21.15.40:0/1631773350 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v49) ==== 137+0+0 (secure 0 0 0) 0x7f43c8016050 con 0x7f43d80ae080 2024-04-02T16:11:57.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 msgr2=0x7f43bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:57.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 0x7f43bc06a840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f43d4007fd0 tx=0x7f43d4009040 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 msgr2=0x7f43d814e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43c80ae040 tx=0x7f43c8019200 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 shutdown_connections 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43d80b2780 0x7f43d806f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f43bc068380 0x7f43bc06a840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43d80b1c10 0x7f43d8071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 --2- 172.21.15.40:0/1631773350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80ae080 0x7f43d814e090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 >> 172.21.15.40:0/1631773350 conn(0x7f43d80b8fd0 msgr2=0x7f43d8153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 shutdown_connections 2024-04-02T16:11:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:57.952+0000 7f43e0409640 1 -- 172.21.15.40:0/1631773350 wait complete. 2024-04-02T16:11:57.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-02T16:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.108+0000 7ffbba7db640 1 Processor -- start 2024-04-02T16:11:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.108+0000 7ffbba7db640 1 -- start start 2024-04-02T16:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbba7db640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbba7db640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 0x7ffbb414ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbba7db640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb40b6310 con 0x7ffbb414b390 2024-04-02T16:11:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbba7db640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb40b55d0 con 0x7ffbb40b3500 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbba7db640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40b3fa0 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 0x7ffbb414ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:54746/0 (socket says 172.21.15.40:54746) 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 0x7ffbb414ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:39164/0 (socket says 172.21.15.40:39164) 2024-04-02T16:11:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 -- 172.21.15.40:0/1000539056 learned_addr learned my addr 172.21.15.40:0/1000539056 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb2ffd640 1 -- 172.21.15.40:0/1000539056 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1478065182 0 0) 0x7ffbb40b6310 con 0x7ffbb414b390 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb2ffd640 1 -- 172.21.15.40:0/1000539056 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb9c003530 con 0x7ffbb414b390 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 -- 172.21.15.40:0/1000539056 >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 msgr2=0x7ffbb414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 --2- 172.21.15.40:0/1000539056 >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 0x7ffbb414ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 -- 172.21.15.40:0/1000539056 >> v1:172.21.15.204:6789/0 conn(0x7ffbb414b390 legacy=0x7ffbb40b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.112+0000 7ffbb3fff640 1 -- 172.21.15.40:0/1000539056 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b4150 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb3fff640 1 --2- 172.21.15.40:0/1000539056 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009870 tx=0x7ffba401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae8b247b27e63cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb2ffd640 1 -- 172.21.15.40:0/1000539056 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba40b0070 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb2ffd640 1 -- 172.21.15.40:0/1000539056 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba40ab600 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb2ffd640 1 -- 172.21.15.40:0/1000539056 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba40ab9b0 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- 172.21.15.40:0/1000539056 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 msgr2=0x7ffbb40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- 172.21.15.40:0/1000539056 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009870 tx=0x7ffba401c440 comp rx=0 tx=0).stop 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- 172.21.15.40:0/1000539056 shutdown_connections 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- 172.21.15.40:0/1000539056 >> v2:172.21.15.204:3300/0 conn(0x7ffbb40b3500 0x7ffbb414ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- 172.21.15.40:0/1000539056 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb40b4fd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- 172.21.15.40:0/1000539056 >> 172.21.15.40:0/1000539056 conn(0x7ffbb40734b0 msgr2=0x7ffbb40bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:58.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- 172.21.15.40:0/1000539056 shutdown_connections 2024-04-02T16:11:58.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- 172.21.15.40:0/1000539056 wait complete. 2024-04-02T16:11:58.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 Processor -- start 2024-04-02T16:11:58.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- start start 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbb40b3500 0x7ffbb4148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb41409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4154c10 con 0x7ffbb414b390 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4154d90 con 0x7ffbb40b3500 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbba7db640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb4154f10 con 0x7ffbb40b4bf0 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb8d78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb8d78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57196/0 (socket says 172.21.15.40:57196) 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb8d78640 1 -- 172.21.15.40:0/2194704827 learned_addr learned my addr 172.21.15.40:0/2194704827 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:58.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb37fe640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb41409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.116+0000 7ffbb3fff640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbb40b3500 0x7ffbb4148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 -- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 msgr2=0x7ffbb41409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb41409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 -- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbb40b3500 msgr2=0x7ffbb4148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbb40b3500 0x7ffbb4148150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 -- 172.21.15.40:0/2194704827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb413e7a0 con 0x7ffbb414b390 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb8d78640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c730 tx=0x7ffbac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac003b80 con 0x7ffbb414b390 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac003d20 con 0x7ffbb414b390 2024-04-02T16:11:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00aa90 con 0x7ffbb414b390 2024-04-02T16:11:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb413fbf0 con 0x7ffbb414b390 2024-04-02T16:11:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb413ea20 con 0x7ffbb414b390 2024-04-02T16:11:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb40685f0 con 0x7ffbb414b390 2024-04-02T16:11:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ffbac00ac30 con 0x7ffbb414b390 2024-04-02T16:11:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 0x7ffba006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffbac086790 con 0x7ffbb414b390 2024-04-02T16:11:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.120+0000 7ffbb3fff640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 0x7ffba006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.124+0000 7ffbb3fff640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 0x7ffba006a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ad000 tx=0x7ffba40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.124+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ffbb413e7a0 con 0x7ffbb414b390 2024-04-02T16:11:58.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.284+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffbb40b88e0 con 0x7ffbb414b390 2024-04-02T16:11:58.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.288+0000 7ffbb0ff9640 1 -- 172.21.15.40:0/2194704827 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ffbac0566f0 con 0x7ffbb414b390 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 msgr2=0x7ffba006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 0x7ffba006a840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ad000 tx=0x7ffba40a6000 comp rx=0 tx=0).stop 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 msgr2=0x7ffbb4137fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c730 tx=0x7ffbac00cbf0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 shutdown_connections 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffbb40b4bf0 0x7ffbb41409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ffba0068380 0x7ffba006a840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb414b390 0x7ffbb4137fd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 --2- 172.21.15.40:0/2194704827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbb40b3500 0x7ffbb4148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 >> 172.21.15.40:0/2194704827 conn(0x7ffbb40734b0 msgr2=0x7ffbb40ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:58.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 shutdown_connections 2024-04-02T16:11:58.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.292+0000 7ffbba7db640 1 -- 172.21.15.40:0/2194704827 wait complete. 2024-04-02T16:11:58.310 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-02T16:11:58.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:11:58.357 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:11:58.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-02T16:11:58.415 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:58.415 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:11:58.420 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:11:58.421 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:11:58.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:11:58.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 Processor -- start 2024-04-02T16:11:58.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 -- start start 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f447406ba60 0x7f4474153010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44740b64a0 con 0x7f447406c290 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4474071060 con 0x7f4474070c80 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f44791b0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f44740b6040 con 0x7f447406ba60 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f447406c290 0x7f447414b3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40538/0 (socket says 172.21.15.40:40538) 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 -- 172.21.15.40:0/1457857480 learned_addr learned my addr 172.21.15.40:0/1457857480 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447359e640 1 --2- 172.21.15.40:0/1457857480 >> v2:172.21.15.204:3300/0 conn(0x7f447406ba60 0x7f4474153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718859633 0 0) 0x7f44740b64a0 con 0x7f447406c290 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4460003530 con 0x7f447406c290 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4085833338 0 0) 0x7f4460003530 con 0x7f447406c290 2024-04-02T16:11:58.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 >> v2:172.21.15.204:3300/0 conn(0x7f447406ba60 msgr2=0x7f4474153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 --2- 172.21.15.40:0/1457857480 >> v2:172.21.15.204:3300/0 conn(0x7f447406ba60 0x7f4474153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 >> v1:172.21.15.204:6789/0 conn(0x7f4474070c80 legacy=0x7f4474072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447414bb00 con 0x7f447406c290 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f445c001680 con 0x7f447406c290 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 >> v1:172.21.15.40:6789/0 conn(0x7f447406c290 legacy=0x7f447414b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44600039c0 0x7f4460003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 0x7f4460004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44600051d0 0x7f446000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b6040 con 0x7f4460004810 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f446000c170 con 0x7f44600051d0 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f446000c370 con 0x7f44600039c0 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 0x7f4460004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447359e640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44600051d0 0x7f446000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4472d9d640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44600039c0 0x7f4460003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 -- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44600039c0 msgr2=0x7f4460003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44600039c0 0x7f4460003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 -- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44600051d0 msgr2=0x7f446000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44600051d0 0x7f446000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447259c640 1 -- 172.21.15.40:0/1457857480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4460003530 con 0x7f4460004810 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f447359e640 1 -- 172.21.15.40:0/1457857480 reap_dead start 2024-04-02T16:11:58.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.632+0000 7f4472d9d640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44600039c0 0x7f4460003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:58.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 0x7f4460004c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c210 tx=0x7f445c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d187816c68695647 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c007d40 con 0x7f4460004810 2024-04-02T16:11:58.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c0a9020 con 0x7f4460004810 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f4471d9b640 1 -- 172.21.15.40:0/1457857480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c005ae0 con 0x7f4460004810 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 msgr2=0x7f4460004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 0x7f4460004c90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c210 tx=0x7f445c01cc60 comp rx=0 tx=0).stop 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- 172.21.15.40:0/1457857480 shutdown_connections 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 --2- 172.21.15.40:0/1457857480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4460004810 0x7f4460004c90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- 172.21.15.40:0/1457857480 >> 172.21.15.40:0/1457857480 conn(0x7f447406e600 msgr2=0x7f447406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- 172.21.15.40:0/1457857480 shutdown_connections 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- 172.21.15.40:0/1457857480 wait complete. 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 Processor -- start 2024-04-02T16:11:58.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- start start 2024-04-02T16:11:58.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4460004810 0x7f44741d76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447413e5b0 0x7f4474138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4474073060 con 0x7f44741400f0 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4474072ee0 con 0x7f447413e5b0 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f44791b0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f44740731e0 con 0x7f4460004810 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57212/0 (socket says 172.21.15.40:57212) 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 -- 172.21.15.40:0/1337932226 learned_addr learned my addr 172.21.15.40:0/1337932226 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 -- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4460004810 msgr2=0x7f44741d76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447359e640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447413e5b0 0x7f4474138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f4472d9d640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4460004810 0x7f44741d76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4460004810 0x7f44741d76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 -- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447413e5b0 msgr2=0x7f4474138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447413e5b0 0x7f4474138fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.636+0000 7f447259c640 1 -- 172.21.15.40:0/1337932226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447413d600 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f447259c640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f445c001980 tx=0x7f4460005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c007d40 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c0b2070 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c004c50 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447413f9f0 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447413db50 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4474068d00 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f445c0ae430 con 0x7f44741400f0 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 0x7f445406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f445c11fe80 con 0x7f44741400f0 2024-04-02T16:11:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f4472d9d640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 0x7f445406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.640+0000 7f4472d9d640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 0x7f445406a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4464000f90 tx=0x7f4464009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.644+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f445c016970 con 0x7f44741400f0 2024-04-02T16:11:58.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.800+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f447413fc60 con 0x7f44741400f0 2024-04-02T16:11:58.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f446b7fe640 1 -- 172.21.15.40:0/1337932226 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f445c0f0550 con 0x7f44741400f0 2024-04-02T16:11:58.805 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:11:58.805 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:11:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 msgr2=0x7f445406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 0x7f445406a840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4464000f90 tx=0x7f4464009040 comp rx=0 tx=0).stop 2024-04-02T16:11:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 msgr2=0x7f447413e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f445c001980 tx=0x7f4460005840 comp rx=0 tx=0).stop 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 shutdown_connections 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4454068380 0x7f445406a840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447413e5b0 0x7f4474138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44741400f0 0x7f447413e070 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 --2- 172.21.15.40:0/1337932226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4460004810 0x7f44741d76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 >> 172.21.15.40:0/1337932226 conn(0x7f447406e600 msgr2=0x7f44740b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 shutdown_connections 2024-04-02T16:11:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.804+0000 7f44791b0640 1 -- 172.21.15.40:0/1337932226 wait complete. 2024-04-02T16:11:58.823 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:11:58.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:11:58.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 Processor -- start 2024-04-02T16:11:58.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 -- start start 2024-04-02T16:11:58.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa15406c8a0 0x7fa1540af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1540ae830 con 0x7fa1540b3800 2024-04-02T16:11:58.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15406cc80 con 0x7fa1540b43a0 2024-04-02T16:11:58.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa15a6ef640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa1540b2b50 con 0x7fa15406c8a0 2024-04-02T16:11:58.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa1537fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa1540b3800 0x7fa1540b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40550/0 (socket says 172.21.15.40:40550) 2024-04-02T16:11:58.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa1537fe640 1 -- 172.21.15.40:0/700559000 learned_addr learned my addr 172.21.15.40:0/700559000 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:58.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa153fff640 1 --2- 172.21.15.40:0/700559000 >> v2:172.21.15.204:3300/0 conn(0x7fa15406c8a0 0x7fa1540af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599230434 0 0) 0x7fa1540ae830 con 0x7fa1540b3800 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa140005740 con 0x7fa1540b3800 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa153fff640 1 -- 172.21.15.40:0/700559000 >> v1:172.21.15.204:6789/0 conn(0x7fa1540b43a0 legacy=0x7fa15414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa153fff640 1 -- 172.21.15.40:0/700559000 >> v1:172.21.15.40:6789/0 conn(0x7fa1540b3800 legacy=0x7fa1540b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 830714235 0 0) 0x7fa140005740 con 0x7fa1540b3800 2024-04-02T16:11:58.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.948+0000 7fa153fff640 1 -- 172.21.15.40:0/700559000 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1540adfa0 con 0x7fa15406c8a0 2024-04-02T16:11:58.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.952+0000 7fa153fff640 1 --2- 172.21.15.40:0/700559000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15406c8a0 0x7fa1540af150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097c0 tx=0x7fa13c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce8c9ae9a7f494e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b0070 con 0x7fa15406c8a0 2024-04-02T16:11:58.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c0ab400 con 0x7fa15406c8a0 2024-04-02T16:11:58.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa152ffd640 1 -- 172.21.15.40:0/700559000 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144001680 con 0x7fa15406c8a0 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 -- 172.21.15.40:0/700559000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15406c8a0 msgr2=0x7fa1540af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/700559000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15406c8a0 0x7fa1540af150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0097c0 tx=0x7fa13c01c330 comp rx=0 tx=0).stop 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 -- 172.21.15.40:0/700559000 shutdown_connections 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/700559000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa15406c8a0 0x7fa1540af150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 -- 172.21.15.40:0/700559000 >> 172.21.15.40:0/700559000 conn(0x7fa1540b8fd0 msgr2=0x7fa1540b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 -- 172.21.15.40:0/700559000 shutdown_connections 2024-04-02T16:11:58.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.956+0000 7fa15a6ef640 1 -- 172.21.15.40:0/700559000 wait complete. 2024-04-02T16:11:58.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 Processor -- start 2024-04-02T16:11:58.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- start start 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1540b3800 0x7fa154146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1540b43a0 0x7fa154146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540b3130 con 0x7fa15406c8a0 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540b54d0 con 0x7fa1540b43a0 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa154146090 con 0x7fa1540b3800 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57226/0 (socket says 172.21.15.40:57226) 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 -- 172.21.15.40:0/2959228925 learned_addr learned my addr 172.21.15.40:0/2959228925 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa1537fe640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1540b3800 0x7fa154146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa158c8c640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1540b43a0 0x7fa154146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 -- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1540b3800 msgr2=0x7fa154146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1540b3800 0x7fa154146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 -- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1540b43a0 msgr2=0x7fa154146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1540b43a0 0x7fa154146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 -- 172.21.15.40:0/2959228925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa154144320 con 0x7fa15406c8a0 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa153fff640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ae040 tx=0x7fa13c01ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b0050 con 0x7fa15406c8a0 2024-04-02T16:11:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13c0bc070 con 0x7fa15406c8a0 2024-04-02T16:11:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b7850 con 0x7fa15406c8a0 2024-04-02T16:11:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa154150500 con 0x7fa15406c8a0 2024-04-02T16:11:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541477d0 con 0x7fa15406c8a0 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540685f0 con 0x7fa15406c8a0 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.960+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa13c0b79f0 con 0x7fa15406c8a0 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.964+0000 7fa150ff9640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 0x7fa13406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.964+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa13c121150 con 0x7fa15406c8a0 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.964+0000 7fa1537fe640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 0x7fa13406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.964+0000 7fa1537fe640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 0x7fa13406a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa15414ff20 tx=0x7fa144009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:58.968+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa13c015fb0 con 0x7fa15406c8a0 2024-04-02T16:11:59.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.124+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1540b32b0 con 0x7fa15406c8a0 2024-04-02T16:11:59.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.124+0000 7fa150ff9640 1 -- 172.21.15.40:0/2959228925 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa13c0f17a0 con 0x7fa15406c8a0 2024-04-02T16:11:59.126 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:11:59.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 msgr2=0x7fa13406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 0x7fa13406a840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa15414ff20 tx=0x7fa144009040 comp rx=0 tx=0).stop 2024-04-02T16:11:59.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 msgr2=0x7fa15414f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ae040 tx=0x7fa13c01ab70 comp rx=0 tx=0).stop 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 shutdown_connections 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1540b3800 0x7fa154146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa134068380 0x7fa13406a840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1540b43a0 0x7fa154146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 --2- 172.21.15.40:0/2959228925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa15406c8a0 0x7fa15414f8c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 >> 172.21.15.40:0/2959228925 conn(0x7fa1540b8fd0 msgr2=0x7fa1540b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 shutdown_connections 2024-04-02T16:11:59.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.128+0000 7fa15a6ef640 1 -- 172.21.15.40:0/2959228925 wait complete. 2024-04-02T16:11:59.144 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:11:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 Processor -- start 2024-04-02T16:11:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 -- start start 2024-04-02T16:11:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 0x7f8de00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de00b19e0 con 0x7f8de0073230 2024-04-02T16:11:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de00b05c0 con 0x7f8de0148e50 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.268+0000 7f8de645d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00b08a0 con 0x7f8de00b23d0 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8de0073230 0x7f8de009daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40566/0 (socket says 172.21.15.40:40566) 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddf7fe640 1 -- 172.21.15.40:0/4024243691 learned_addr learned my addr 172.21.15.40:0/4024243691 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddffff640 1 --2- 172.21.15.40:0/4024243691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 0x7f8de00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 540630497 0 0) 0x7f8de00b19e0 con 0x7f8de0073230 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dcc003530 con 0x7f8de0073230 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952251502 0 0) 0x7f8de00b05c0 con 0x7f8de0148e50 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de00b19e0 con 0x7f8de0148e50 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2645760570 0 0) 0x7f8dcc003530 con 0x7f8de0073230 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddffff640 1 -- 172.21.15.40:0/4024243691 >> v1:172.21.15.204:6789/0 conn(0x7f8de0148e50 legacy=0x7f8de01492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddffff640 1 -- 172.21.15.40:0/4024243691 >> v1:172.21.15.40:6789/0 conn(0x7f8de0073230 legacy=0x7f8de009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddffff640 1 -- 172.21.15.40:0/4024243691 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de00b0b80 con 0x7f8de00b23d0 2024-04-02T16:11:59.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddffff640 1 --2- 172.21.15.40:0/4024243691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 0x7f8de00b27b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00097f0 tx=0x7f8dd001c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a15f32965f101b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00b0070 con 0x7f8de00b23d0 2024-04-02T16:11:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd00ab400 con 0x7f8de00b23d0 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8ddeffd640 1 -- 172.21.15.40:0/4024243691 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8001680 con 0x7f8de00b23d0 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8de645d640 1 -- 172.21.15.40:0/4024243691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 msgr2=0x7f8de00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.272+0000 7f8de645d640 1 --2- 172.21.15.40:0/4024243691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 0x7f8de00b27b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00097f0 tx=0x7f8dd001c370 comp rx=0 tx=0).stop 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/4024243691 shutdown_connections 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 --2- 172.21.15.40:0/4024243691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de00b23d0 0x7f8de00b27b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/4024243691 >> 172.21.15.40:0/4024243691 conn(0x7f8de00b8fd0 msgr2=0x7f8de00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/4024243691 shutdown_connections 2024-04-02T16:11:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/4024243691 wait complete. 2024-04-02T16:11:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 Processor -- start 2024-04-02T16:11:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- start start 2024-04-02T16:11:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de0073230 0x7f8de014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 0x7f8de013f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0153a30 con 0x7f8de0148e50 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01538b0 con 0x7f8de00b23d0 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8de0153bb0 con 0x7f8de0073230 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57234/0 (socket says 172.21.15.40:57234) 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 -- 172.21.15.40:0/3140842527 learned_addr learned my addr 172.21.15.40:0/3140842527 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8ddf7fe640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 0x7f8de013f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 -- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de0073230 msgr2=0x7f8de014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de0073230 0x7f8de014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 -- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 msgr2=0x7f8de013f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 0x7f8de013f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 -- 172.21.15.40:0/3140842527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de014ff20 con 0x7f8de0148e50 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8ddf7fe640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 0x7f8de013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:11:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de49fa640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c730 tx=0x7f8dd800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8003b80 con 0x7f8de0148e50 2024-04-02T16:11:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd8003d20 con 0x7f8de0148e50 2024-04-02T16:11:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de014e8e0 con 0x7f8de0148e50 2024-04-02T16:11:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800aa90 con 0x7f8de0148e50 2024-04-02T16:11:59.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de00af040 con 0x7f8de0148e50 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.276+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00685f0 con 0x7f8de0148e50 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.280+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8dd800ac30 con 0x7f8de0148e50 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.280+0000 7f8ddcff9640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 0x7f8dc006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.280+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8dd8086830 con 0x7f8de0148e50 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.280+0000 7f8ddffff640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 0x7f8dc006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.280+0000 7f8ddffff640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 0x7f8dc006a870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00ad000 tx=0x7f8dd0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.284+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f8dd8056e80 con 0x7f8de0148e50 2024-04-02T16:11:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.444+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8de0068f20 con 0x7f8de0148e50 2024-04-02T16:11:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.444+0000 7f8ddcff9640 1 -- 172.21.15.40:0/3140842527 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8dd8056780 con 0x7f8de0148e50 2024-04-02T16:11:59.447 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:11:59.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 msgr2=0x7f8dc006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 0x7f8dc006a870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00ad000 tx=0x7f8dd0004740 comp rx=0 tx=0).stop 2024-04-02T16:11:59.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 msgr2=0x7f8de01e3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c730 tx=0x7f8dd800cbf0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 shutdown_connections 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8dc00683b0 0x7f8dc006a870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8de0148e50 0x7f8de01e3990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de00b23d0 0x7f8de013f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 --2- 172.21.15.40:0/3140842527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de0073230 0x7f8de014d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 >> 172.21.15.40:0/3140842527 conn(0x7f8de00b8fd0 msgr2=0x7f8de006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 shutdown_connections 2024-04-02T16:11:59.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.448+0000 7f8de645d640 1 -- 172.21.15.40:0/3140842527 wait complete. 2024-04-02T16:11:59.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:11:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 Processor -- start 2024-04-02T16:11:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 -- start start 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa88006c8a0 0x7fa8800af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8800ae830 con 0x7fa8800b3800 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa88006cc80 con 0x7fa8800b43a0 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa8885c7640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa8800b2b50 con 0x7fa88006c8a0 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8800b3800 0x7fa8800b3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40578/0 (socket says 172.21.15.40:40578) 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 -- 172.21.15.40:0/3271788413 learned_addr learned my addr 172.21.15.40:0/3271788413 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa886363640 1 --2- 172.21.15.40:0/3271788413 >> v2:172.21.15.204:3300/0 conn(0x7fa88006c8a0 0x7fa8800af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182391366 0 0) 0x7fa8800ae830 con 0x7fa8800b3800 2024-04-02T16:11:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa868003530 con 0x7fa8800b3800 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916293930 0 0) 0x7fa868003530 con 0x7fa8800b3800 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 >> v2:172.21.15.204:3300/0 conn(0x7fa88006c8a0 msgr2=0x7fa8800af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 --2- 172.21.15.40:0/3271788413 >> v2:172.21.15.204:3300/0 conn(0x7fa88006c8a0 0x7fa8800af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 >> v1:172.21.15.204:6789/0 conn(0x7fa8800b43a0 legacy=0x7fa88014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8800adfa0 con 0x7fa8800b3800 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fa874001680 con 0x7fa8800b3800 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 >> v1:172.21.15.40:6789/0 conn(0x7fa8800b3800 legacy=0x7fa8800b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8680039c0 0x7fa868003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 0x7fa868004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680051d0 0x7fa86800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800b2b50 con 0x7fa868004810 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa86800c170 con 0x7fa8680039c0 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa86800c370 con 0x7fa8680051d0 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 0x7fa868004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa886363640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8680039c0 0x7fa868003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa886b64640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680051d0 0x7fa86800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 -- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680051d0 msgr2=0x7fa86800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680051d0 0x7fa86800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 -- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8680039c0 msgr2=0x7fa868003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8680039c0 0x7fa868003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa885b62640 1 -- 172.21.15.40:0/3271788413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa868003530 con 0x7fa868004810 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa886b64640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8680051d0 0x7fa86800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.588+0000 7fa886b64640 1 -- 172.21.15.40:0/3271788413 reap_dead start 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa885b62640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 0x7fa868004c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c240 tx=0x7fa87401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f77f4d29b28fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874007d20 con 0x7fa868004810 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8740a9070 con 0x7fa868004810 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa885361640 1 -- 172.21.15.40:0/3271788413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874005d80 con 0x7fa868004810 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 msgr2=0x7fa868004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 0x7fa868004c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c240 tx=0x7fa87401cc80 comp rx=0 tx=0).stop 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- 172.21.15.40:0/3271788413 shutdown_connections 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 --2- 172.21.15.40:0/3271788413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa868004810 0x7fa868004c90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- 172.21.15.40:0/3271788413 >> 172.21.15.40:0/3271788413 conn(0x7fa8800b8fd0 msgr2=0x7fa8800b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- 172.21.15.40:0/3271788413 shutdown_connections 2024-04-02T16:11:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- 172.21.15.40:0/3271788413 wait complete. 2024-04-02T16:11:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 Processor -- start 2024-04-02T16:11:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.592+0000 7fa8885c7640 1 -- start start 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 0x7fa880148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8801d1630 0x7fa8801d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800b41e0 con 0x7fa88006f880 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8800b4060 con 0x7fa868004810 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa8800b54d0 con 0x7fa8801d1630 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57248/0 (socket says 172.21.15.40:57248) 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 -- 172.21.15.40:0/1389814689 learned_addr learned my addr 172.21.15.40:0/1389814689 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa885b62640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8801d1630 0x7fa8801d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886363640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 0x7fa880148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 -- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8801d1630 msgr2=0x7fa8801d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8801d1630 0x7fa8801d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 -- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 msgr2=0x7fa880148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 0x7fa880148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 -- 172.21.15.40:0/1389814689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801d3f70 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886363640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 0x7fa880148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886b64640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa88006e1e0 tx=0x7fa87000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870018070 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870003600 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801d35d0 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8700039f0 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8800714e0 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8800685f0 con 0x7fa88006f880 2024-04-02T16:11:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa87001b070 con 0x7fa88006f880 2024-04-02T16:11:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 0x7fa86c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa870085e50 con 0x7fa88006f880 2024-04-02T16:11:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.596+0000 7fa886363640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 0x7fa86c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.600+0000 7fa886363640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 0x7fa86c06a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa878007440 tx=0x7fa87800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.600+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa8700563f0 con 0x7fa88006f880 2024-04-02T16:11:59.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.756+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa8801d3d40 con 0x7fa88006f880 2024-04-02T16:11:59.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa87effd640 1 -- 172.21.15.40:0/1389814689 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa870055cf0 con 0x7fa88006f880 2024-04-02T16:11:59.760 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:11:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 msgr2=0x7fa86c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 0x7fa86c06a840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa878007440 tx=0x7fa87800b040 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 msgr2=0x7fa88006fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa88006e1e0 tx=0x7fa87000d890 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 shutdown_connections 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8801d1630 0x7fa8801d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa86c068380 0x7fa86c06a840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88006f880 0x7fa88006fd00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 --2- 172.21.15.40:0/1389814689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa868004810 0x7fa880148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 >> 172.21.15.40:0/1389814689 conn(0x7fa8800b8fd0 msgr2=0x7fa880147430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 shutdown_connections 2024-04-02T16:11:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.760+0000 7fa8885c7640 1 -- 172.21.15.40:0/1389814689 wait complete. 2024-04-02T16:11:59.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:11:59.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 Processor -- start 2024-04-02T16:11:59.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 -- start start 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc80b23d0 0x7fabc80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fabc8073230 0x7fabc809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80b19e0 con 0x7fabc809dfe0 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fabc80b05c0 con 0x7fabc8073230 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcef51640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80b08a0 con 0x7fabc80b23d0 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcd4ee640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fabc809dfe0 0x7fabc8148e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40582/0 (socket says 172.21.15.40:40582) 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabcd4ee640 1 -- 172.21.15.40:0/2960270234 learned_addr learned my addr 172.21.15.40:0/2960270234 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:11:59.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc7fff640 1 --2- 172.21.15.40:0/2960270234 >> v2:172.21.15.40:3300/0 conn(0x7fabc8073230 0x7fabc809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabccced640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc80b23d0 0x7fabc80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc77fe640 1 -- 172.21.15.40:0/2960270234 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188637565 0 0) 0x7fabc80b19e0 con 0x7fabc809dfe0 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc77fe640 1 -- 172.21.15.40:0/2960270234 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabb0003530 con 0x7fabc809dfe0 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc7fff640 1 -- 172.21.15.40:0/2960270234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc80b23d0 msgr2=0x7fabc80b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc7fff640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc80b23d0 0x7fabc80b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc7fff640 1 -- 172.21.15.40:0/2960270234 >> v1:172.21.15.40:6789/0 conn(0x7fabc809dfe0 legacy=0x7fabc8148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.904+0000 7fabc7fff640 1 -- 172.21.15.40:0/2960270234 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80b0b80 con 0x7fabc8073230 2024-04-02T16:11:59.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabc7fff640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc8073230 0x7fabc809daa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097c0 tx=0x7fabbc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a9027cd7247927 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabc77fe640 1 -- 172.21.15.40:0/2960270234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0b0070 con 0x7fabc8073230 2024-04-02T16:11:59.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabc77fe640 1 -- 172.21.15.40:0/2960270234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0ab600 con 0x7fabc8073230 2024-04-02T16:11:59.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabc77fe640 1 -- 172.21.15.40:0/2960270234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0ab9b0 con 0x7fabc8073230 2024-04-02T16:11:59.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- 172.21.15.40:0/2960270234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc8073230 msgr2=0x7fabc809daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc8073230 0x7fabc809daa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097c0 tx=0x7fabbc01c410 comp rx=0 tx=0).stop 2024-04-02T16:11:59.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- 172.21.15.40:0/2960270234 shutdown_connections 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc8073230 0x7fabc809daa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- 172.21.15.40:0/2960270234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc80b23d0 0x7fabc80b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- 172.21.15.40:0/2960270234 >> 172.21.15.40:0/2960270234 conn(0x7fabc80b8fd0 msgr2=0x7fabc80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- 172.21.15.40:0/2960270234 shutdown_connections 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- 172.21.15.40:0/2960270234 wait complete. 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 Processor -- start 2024-04-02T16:11:59.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- start start 2024-04-02T16:11:59.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabc8073230 0x7fabc8147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc809dfe0 0x7fabc81375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8152a20 con 0x7fabc80b23d0 2024-04-02T16:11:59.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81528a0 con 0x7fabc8073230 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcef51640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fabc8152ba0 con 0x7fabc809dfe0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcd4ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcd4ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57266/0 (socket says 172.21.15.40:57266) 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.908+0000 7fabcd4ee640 1 -- 172.21.15.40:0/2936602070 learned_addr learned my addr 172.21.15.40:0/2936602070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc7fff640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc809dfe0 0x7fabc81375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabccced640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabc8073230 0x7fabc8147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 -- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc809dfe0 msgr2=0x7fabc81375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc809dfe0 0x7fabc81375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 -- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabc8073230 msgr2=0x7fabc8147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabc8073230 0x7fabc8147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 -- 172.21.15.40:0/2936602070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc8137c90 con 0x7fabc80b23d0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcd4ee640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c480 tx=0x7fabc000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0017070 con 0x7fabc80b23d0 2024-04-02T16:11:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc0005cf0 con 0x7fabc80b23d0 2024-04-02T16:11:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc8148220 con 0x7fabc80b23d0 2024-04-02T16:11:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc00072b0 con 0x7fabc80b23d0 2024-04-02T16:11:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc8140b70 con 0x7fabc80b23d0 2024-04-02T16:11:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc80685f0 con 0x7fabc80b23d0 2024-04-02T16:11:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fabc001a070 con 0x7fabc80b23d0 2024-04-02T16:11:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 0x7fabb406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:11:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabc0085d30 con 0x7fabc80b23d0 2024-04-02T16:11:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabccced640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 0x7fabb406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:11:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.912+0000 7fabccced640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 0x7fabb406a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007380 tx=0x7fabb800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:11:59.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:11:59.916+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fabc00562e0 con 0x7fabc80b23d0 2024-04-02T16:12:00.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.092+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fabc81488e0 con 0x7fabc80b23d0 2024-04-02T16:12:00.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.092+0000 7fabc57fa640 1 -- 172.21.15.40:0/2936602070 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fabc0055be0 con 0x7fabc80b23d0 2024-04-02T16:12:00.095 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-02T16:12:00.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 msgr2=0x7fabb406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 0x7fabb406a840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabb8007380 tx=0x7fabb800b040 comp rx=0 tx=0).stop 2024-04-02T16:12:00.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 msgr2=0x7fabc8140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c480 tx=0x7fabc000c940 comp rx=0 tx=0).stop 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 shutdown_connections 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabc809dfe0 0x7fabc81375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fabb4068380 0x7fabb406a840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc80b23d0 0x7fabc8140630 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 --2- 172.21.15.40:0/2936602070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabc8073230 0x7fabc8147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 >> 172.21.15.40:0/2936602070 conn(0x7fabc80b8fd0 msgr2=0x7fabc80b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 shutdown_connections 2024-04-02T16:12:00.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.096+0000 7fabcef51640 1 -- 172.21.15.40:0/2936602070 wait complete. 2024-04-02T16:12:00.115 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:12:00.115 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:12:00.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.284+0000 7f25b7396640 1 Processor -- start 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.284+0000 7f25b7396640 1 -- start start 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b7396640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25b00b23d0 0x7f25b00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b7396640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 0x7f25b0149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b7396640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b00b19e0 con 0x7f25b0073230 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b7396640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f25b01497c0 con 0x7f25b00b23d0 2024-04-02T16:12:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b7396640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25b00b05c0 con 0x7f25b0148e50 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25b00b23d0 0x7f25b00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25b00b23d0 0x7f25b00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57282/0 (socket says 172.21.15.40:57282) 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 -- 172.21.15.40:0/4230802469 learned_addr learned my addr 172.21.15.40:0/4230802469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5933640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 0x7f25b0149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 -- 172.21.15.40:0/4230802469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 msgr2=0x7f25b0149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 0x7f25b0149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 -- 172.21.15.40:0/4230802469 >> v1:172.21.15.40:6789/0 conn(0x7f25b0073230 legacy=0x7f25b009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25affff640 1 -- 172.21.15.40:0/4230802469 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1279612549 0 0) 0x7f25b00b19e0 con 0x7f25b0073230 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 -- 172.21.15.40:0/4230802469 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00b08a0 con 0x7f25b00b23d0 2024-04-02T16:12:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5933640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 0x7f25b0149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25b5132640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b00b27b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009e50 tx=0x7f25a801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327063d417930e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25affff640 1 -- 172.21.15.40:0/4230802469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80a5880 con 0x7f25b00b23d0 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25affff640 1 -- 172.21.15.40:0/4230802469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a80a5a20 con 0x7f25b00b23d0 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.288+0000 7f25affff640 1 -- 172.21.15.40:0/4230802469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a80a5df0 con 0x7f25b00b23d0 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/4230802469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 msgr2=0x7f25b00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b00b27b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009e50 tx=0x7f25a801c9a0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/4230802469 shutdown_connections 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0148e50 0x7f25b0149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- 172.21.15.40:0/4230802469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b00b27b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/4230802469 >> 172.21.15.40:0/4230802469 conn(0x7f25b00b8fd0 msgr2=0x7f25b00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/4230802469 shutdown_connections 2024-04-02T16:12:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/4230802469 wait complete. 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 Processor -- start 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- start start 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 0x7f25b01e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 0x7f25b00af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0152810 con 0x7f25b00b23d0 2024-04-02T16:12:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0152690 con 0x7f25b0148e50 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0152990 con 0x7f25b0073230 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57290/0 (socket says 172.21.15.40:57290) 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 -- 172.21.15.40:0/3098925909 learned_addr learned my addr 172.21.15.40:0/3098925909 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b5132640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 0x7f25b01e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b5933640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 0x7f25b00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 -- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 msgr2=0x7f25b01e3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 0x7f25b01e3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 -- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 msgr2=0x7f25b00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 0x7f25b00af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 -- 172.21.15.40:0/3098925909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00ae820 con 0x7f25b00b23d0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b5132640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 0x7f25b01e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b5933640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 0x7f25b00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b4931640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c4d0 tx=0x7f25a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4017070 con 0x7f25b00b23d0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a4005480 con 0x7f25b00b23d0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b009c390 con 0x7f25b00b23d0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4003840 con 0x7f25b00b23d0 2024-04-02T16:12:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b009b260 con 0x7f25b00b23d0 2024-04-02T16:12:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.292+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b00685f0 con 0x7f25b00b23d0 2024-04-02T16:12:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.296+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f25a401a070 con 0x7f25b00b23d0 2024-04-02T16:12:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.296+0000 7f25adffb640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 0x7f259c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.296+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f25a4085db0 con 0x7f25b00b23d0 2024-04-02T16:12:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.296+0000 7f25b5132640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 0x7f259c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.296+0000 7f25b5132640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 0x7f259c06a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25a80ae000 tx=0x7f25a80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.300+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f25a4056430 con 0x7f25b00b23d0 2024-04-02T16:12:00.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.460+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25b00b56e0 con 0x7f25b00b23d0 2024-04-02T16:12:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25adffb640 1 -- 172.21.15.40:0/3098925909 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f25a4055d30 con 0x7f25b00b23d0 2024-04-02T16:12:00.464 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:00.464 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 msgr2=0x7f259c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 0x7f259c06a840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f25a80ae000 tx=0x7f25a80a7070 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 msgr2=0x7f25b01e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c4d0 tx=0x7f25a400c990 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 shutdown_connections 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f259c068380 0x7f259c06a840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25b0148e50 0x7f25b00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25b00b23d0 0x7f25b01e4160 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 --2- 172.21.15.40:0/3098925909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25b0073230 0x7f25b01e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 >> 172.21.15.40:0/3098925909 conn(0x7f25b00b8fd0 msgr2=0x7f25b006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 shutdown_connections 2024-04-02T16:12:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.464+0000 7f25b7396640 1 -- 172.21.15.40:0/3098925909 wait complete. 2024-04-02T16:12:00.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:12:00.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:12:00.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 Processor -- start 2024-04-02T16:12:00.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 -- start start 2024-04-02T16:12:00.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f024c06c8a0 0x7f024c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b3800 0x7f024c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f024c0ae830 con 0x7f024c0b41a0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f024c0b2b50 con 0x7f024c06c8a0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f0252b20640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f024c0b0300 con 0x7f024c0b3800 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02508bc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f024c06c8a0 0x7f024c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02510bd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f024c0b41a0 0x7f024c147c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40608/0 (socket says 172.21.15.40:40608) 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02510bd640 1 -- 172.21.15.40:0/856748738 learned_addr learned my addr 172.21.15.40:0/856748738 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f024bfff640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b3800 0x7f024c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521544560 0 0) 0x7f024c0ae830 con 0x7f024c0b41a0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023c003530 con 0x7f024c0b41a0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02508bc640 1 -- 172.21.15.40:0/856748738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b3800 msgr2=0x7f024c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02508bc640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b3800 0x7f024c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4179031770 0 0) 0x7f023c003530 con 0x7f024c0b41a0 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02508bc640 1 -- 172.21.15.40:0/856748738 >> v1:172.21.15.40:6789/0 conn(0x7f024c0b41a0 legacy=0x7f024c147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.612+0000 7f02508bc640 1 -- 172.21.15.40:0/856748738 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c0b4610 con 0x7f024c06c8a0 2024-04-02T16:12:00.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02508bc640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c06c8a0 0x7f024c0af150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02400097c0 tx=0x7f024001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d101872032bc331 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0070 con 0x7f024c06c8a0 2024-04-02T16:12:00.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02400ab400 con 0x7f024c06c8a0 2024-04-02T16:12:00.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024b7fe640 1 -- 172.21.15.40:0/856748738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244001a20 con 0x7f024c06c8a0 2024-04-02T16:12:00.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/856748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c06c8a0 msgr2=0x7f024c0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c06c8a0 0x7f024c0af150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f02400097c0 tx=0x7f024001c2a0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/856748738 shutdown_connections 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b3800 0x7f024c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- 172.21.15.40:0/856748738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c06c8a0 0x7f024c0af150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/856748738 >> 172.21.15.40:0/856748738 conn(0x7f024c0b8fd0 msgr2=0x7f024c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/856748738 shutdown_connections 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/856748738 wait complete. 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 Processor -- start 2024-04-02T16:12:00.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- start start 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f024c06c8a0 0x7f024c143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b41a0 0x7f024c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f024c0af690 con 0x7f024c0b3800 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f024c0b0540 con 0x7f024c06c8a0 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f024c0b3130 con 0x7f024c0b41a0 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57298/0 (socket says 172.21.15.40:57298) 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 -- 172.21.15.40:0/1418681152 learned_addr learned my addr 172.21.15.40:0/1418681152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02510bd640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b41a0 0x7f024c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02508bc640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f024c06c8a0 0x7f024c143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 -- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b41a0 msgr2=0x7f024c06f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b41a0 0x7f024c06f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 -- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f024c06c8a0 msgr2=0x7f024c143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f024c06c8a0 0x7f024c143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 -- 172.21.15.40:0/1418681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c071490 con 0x7f024c0b3800 2024-04-02T16:12:00.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f024bfff640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f023800e930 tx=0x7f023800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238017070 con 0x7f024c0b3800 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f023800ca00 con 0x7f024c0b3800 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c071770 con 0x7f024c0b3800 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800cdb0 con 0x7f024c0b3800 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.616+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c070b30 con 0x7f024c0b3800 2024-04-02T16:12:00.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024c13cbf0 con 0x7f024c0b3800 2024-04-02T16:12:00.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f023801a070 con 0x7f024c0b3800 2024-04-02T16:12:00.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f02497fa640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 0x7f023006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0238086c30 con 0x7f024c0b3800 2024-04-02T16:12:00.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f02508bc640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 0x7f023006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.620+0000 7f02508bc640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 0x7f023006a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02400ad000 tx=0x7f02400a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.624+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f0238056cf0 con 0x7f024c0b3800 2024-04-02T16:12:00.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.784+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f024c0726e0 con 0x7f024c0b3800 2024-04-02T16:12:00.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f02497fa640 1 -- 172.21.15.40:0/1418681152 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f024c0726e0 con 0x7f024c0b3800 2024-04-02T16:12:00.788 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:12:00.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 msgr2=0x7f023006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 0x7f023006a870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02400ad000 tx=0x7f02400a6000 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 msgr2=0x7f024c071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f023800e930 tx=0x7f023800edf0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 shutdown_connections 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f024c0b41a0 0x7f024c06f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f02300683b0 0x7f023006a870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024c0b3800 0x7f024c071ef0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 --2- 172.21.15.40:0/1418681152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f024c06c8a0 0x7f024c143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 >> 172.21.15.40:0/1418681152 conn(0x7f024c0b8fd0 msgr2=0x7f024c0b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 shutdown_connections 2024-04-02T16:12:00.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.788+0000 7f0252b20640 1 -- 172.21.15.40:0/1418681152 wait complete. 2024-04-02T16:12:00.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:12:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 Processor -- start 2024-04-02T16:12:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 -- start start 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00b23d0 0x7f64d00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d0073230 0x7f64d009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64d00b19e0 con 0x7f64d009dfe0 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f64d00b05c0 con 0x7f64d00b23d0 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64d5871640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64d00b08a0 con 0x7f64d0073230 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00b23d0 0x7f64d00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00b23d0 0x7f64d00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57300/0 (socket says 172.21.15.40:57300) 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64cf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64d009dfe0 0x7f64d0148e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40618/0 (socket says 172.21.15.40:40618) 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 -- 172.21.15.40:0/1835244818 learned_addr learned my addr 172.21.15.40:0/1835244818 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 -- 172.21.15.40:0/1835244818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d0073230 msgr2=0x7f64d009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 --2- 172.21.15.40:0/1835244818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d0073230 0x7f64d009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 -- 172.21.15.40:0/1835244818 >> v1:172.21.15.40:6789/0 conn(0x7f64d009dfe0 legacy=0x7f64d0148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.932+0000 7f64ceffd640 1 -- 172.21.15.40:0/1835244818 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d00b0b80 con 0x7f64d00b23d0 2024-04-02T16:12:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64ceffd640 1 --2- 172.21.15.40:0/1835244818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00b23d0 0x7f64d00b27b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009e50 tx=0x7f64c801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f233aaa6f34693d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64cdffb640 1 -- 172.21.15.40:0/1835244818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80a5780 con 0x7f64d00b23d0 2024-04-02T16:12:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64cdffb640 1 -- 172.21.15.40:0/1835244818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c80a5920 con 0x7f64d00b23d0 2024-04-02T16:12:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64cdffb640 1 -- 172.21.15.40:0/1835244818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80a5cf0 con 0x7f64d00b23d0 2024-04-02T16:12:00.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- 172.21.15.40:0/1835244818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00b23d0 msgr2=0x7f64d00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 --2- 172.21.15.40:0/1835244818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00b23d0 0x7f64d00b27b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009e50 tx=0x7f64c801c9a0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- 172.21.15.40:0/1835244818 shutdown_connections 2024-04-02T16:12:00.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 --2- 172.21.15.40:0/1835244818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d0073230 0x7f64d009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 --2- 172.21.15.40:0/1835244818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00b23d0 0x7f64d00b27b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- 172.21.15.40:0/1835244818 >> 172.21.15.40:0/1835244818 conn(0x7f64d00b8fd0 msgr2=0x7f64d00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- 172.21.15.40:0/1835244818 shutdown_connections 2024-04-02T16:12:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- 172.21.15.40:0/1835244818 wait complete. 2024-04-02T16:12:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 Processor -- start 2024-04-02T16:12:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.936+0000 7f64d5871640 1 -- start start 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64d009dfe0 0x7f64d0135330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d00b23d0 0x7f64d0135870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0152a20 con 0x7f64d0073230 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01528a0 con 0x7f64d009dfe0 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64d0152ba0 con 0x7f64d00b23d0 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57306/0 (socket says 172.21.15.40:57306) 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 -- 172.21.15.40:0/978397617 learned_addr learned my addr 172.21.15.40:0/978397617 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 -- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d00b23d0 msgr2=0x7f64d0135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64cf7fe640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d00b23d0 0x7f64d0135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ce7fc640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64d009dfe0 0x7f64d0135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d00b23d0 0x7f64d0135870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 -- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64d009dfe0 msgr2=0x7f64d0135330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64d009dfe0 0x7f64d0135330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 -- 172.21.15.40:0/978397617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d009d080 con 0x7f64d0073230 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64ceffd640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64c801a5d0 tx=0x7f64c8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80bc020 con 0x7f64d0073230 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c80b88c0 con 0x7f64d0073230 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d009d670 con 0x7f64d0073230 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80b8cb0 con 0x7f64d0073230 2024-04-02T16:12:00.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d013edb0 con 0x7f64d0073230 2024-04-02T16:12:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d00685f0 con 0x7f64d0073230 2024-04-02T16:12:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f64c80bf020 con 0x7f64d0073230 2024-04-02T16:12:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 0x7f64bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.940+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f64c8122a50 con 0x7f64d0073230 2024-04-02T16:12:00.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.944+0000 7f64ce7fc640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 0x7f64bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:00.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.944+0000 7f64ce7fc640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 0x7f64bc06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64d0146890 tx=0x7f64c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:00.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:00.944+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f64d009d080 con 0x7f64d0073230 2024-04-02T16:12:01.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.108+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64d009d3c0 con 0x7f64d0073230 2024-04-02T16:12:01.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.108+0000 7f64b7fff640 1 -- 172.21.15.40:0/978397617 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f64c8016620 con 0x7f64d0073230 2024-04-02T16:12:01.110 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 msgr2=0x7f64bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 0x7f64bc06a840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64d0146890 tx=0x7f64c4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 msgr2=0x7f64d013f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f64c801a5d0 tx=0x7f64c8019740 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 shutdown_connections 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64d00b23d0 0x7f64d0135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f64bc068380 0x7f64bc06a840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64d009dfe0 0x7f64d0135330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 --2- 172.21.15.40:0/978397617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0073230 0x7f64d013f530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 >> 172.21.15.40:0/978397617 conn(0x7f64d00b8fd0 msgr2=0x7f64d006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:01.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 shutdown_connections 2024-04-02T16:12:01.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.112+0000 7f64d5871640 1 -- 172.21.15.40:0/978397617 wait complete. 2024-04-02T16:12:01.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:12:01.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.256+0000 7f2280ffa640 1 Processor -- start 2024-04-02T16:12:01.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.256+0000 7f2280ffa640 1 -- start start 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f227c149560 0x7f227c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227c0b19e0 con 0x7f227c0b23d0 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f227c0b05c0 con 0x7f227c073230 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f227c0b08a0 con 0x7f227c149560 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f227c0b23d0 0x7f227c09daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40630/0 (socket says 172.21.15.40:40630) 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227a59c640 1 -- 172.21.15.40:0/699859190 learned_addr learned my addr 172.21.15.40:0/699859190 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227ad9d640 1 --2- 172.21.15.40:0/699859190 >> v2:172.21.15.40:3300/0 conn(0x7f227c149560 0x7f227c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227ad9d640 1 -- 172.21.15.40:0/699859190 >> v1:172.21.15.204:6789/0 conn(0x7f227c073230 legacy=0x7f227c148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227ad9d640 1 -- 172.21.15.40:0/699859190 >> v1:172.21.15.40:6789/0 conn(0x7f227c0b23d0 legacy=0x7f227c09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227ad9d640 1 -- 172.21.15.40:0/699859190 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c0b0b80 con 0x7f227c149560 2024-04-02T16:12:01.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227ad9d640 1 --2- 172.21.15.40:0/699859190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c149560 0x7f227c0b27b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54af084e0bd9ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227959a640 1 -- 172.21.15.40:0/699859190 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740b2070 con 0x7f227c149560 2024-04-02T16:12:01.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227959a640 1 -- 172.21.15.40:0/699859190 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22740a5880 con 0x7f227c149560 2024-04-02T16:12:01.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f227959a640 1 -- 172.21.15.40:0/699859190 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740a5c50 con 0x7f227c149560 2024-04-02T16:12:01.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- 172.21.15.40:0/699859190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c149560 msgr2=0x7f227c0b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 --2- 172.21.15.40:0/699859190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c149560 0x7f227c0b27b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227401c8a0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- 172.21.15.40:0/699859190 shutdown_connections 2024-04-02T16:12:01.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 --2- 172.21.15.40:0/699859190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c149560 0x7f227c0b27b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- 172.21.15.40:0/699859190 >> 172.21.15.40:0/699859190 conn(0x7f227c0b8fd0 msgr2=0x7f227c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:01.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- 172.21.15.40:0/699859190 shutdown_connections 2024-04-02T16:12:01.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.260+0000 7f2280ffa640 1 -- 172.21.15.40:0/699859190 wait complete. 2024-04-02T16:12:01.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 Processor -- start 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- start start 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227c0b23d0 0x7f227c13fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f227c149560 0x7f227c142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f227c153a30 con 0x7f227c073230 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f227c1538b0 con 0x7f227c149560 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f227c153bb0 con 0x7f227c0b23d0 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57328/0 (socket says 172.21.15.40:57328) 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 -- 172.21.15.40:0/991853391 learned_addr learned my addr 172.21.15.40:0/991853391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 -- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227c0b23d0 msgr2=0x7f227c13fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227c0b23d0 0x7f227c13fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 -- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f227c149560 msgr2=0x7f227c142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f227c149560 0x7f227c142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 -- 172.21.15.40:0/991853391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c09cef0 con 0x7f227c073230 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f227a59c640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f227000c420 tx=0x7f227000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270017070 con 0x7f227c073230 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270005cf0 con 0x7f227c073230 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227c140770 con 0x7f227c073230 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700072d0 con 0x7f227c073230 2024-04-02T16:12:01.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f227c140a50 con 0x7f227c073230 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f227c0685f0 con 0x7f227c073230 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f227001a070 con 0x7f227c073230 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.264+0000 7f226affd640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 0x7f225c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.268+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2270085bb0 con 0x7f227c073230 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.268+0000 7f2279d9b640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 0x7f225c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.268+0000 7f2279d9b640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 0x7f225c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f227c09c640 tx=0x7f226c0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.272+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2270055bb0 con 0x7f227c073230 2024-04-02T16:12:01.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.424+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f227c0b56e0 con 0x7f227c073230 2024-04-02T16:12:01.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f226affd640 1 -- 172.21.15.40:0/991853391 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f2270059eb0 con 0x7f227c073230 2024-04-02T16:12:01.428 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 msgr2=0x7f225c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 0x7f225c06a840 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f227c09c640 tx=0x7f226c0079c0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 msgr2=0x7f227c13b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f227000c420 tx=0x7f227000c8e0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 shutdown_connections 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f227c0b23d0 0x7f227c13fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f225c068380 0x7f225c06a840 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f227c149560 0x7f227c142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 --2- 172.21.15.40:0/991853391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227c073230 0x7f227c13b190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 >> 172.21.15.40:0/991853391 conn(0x7f227c0b8fd0 msgr2=0x7f227c0b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:01.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 shutdown_connections 2024-04-02T16:12:01.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.428+0000 7f2280ffa640 1 -- 172.21.15.40:0/991853391 wait complete. 2024-04-02T16:12:01.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:12:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 Processor -- start 2024-04-02T16:12:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 -- start start 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f925c06c8a0 0x7f925c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0ae830 con 0x7f925c0b43a0 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c06cc80 con 0x7f925c0b3800 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926328f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f925c0b2b50 con 0x7f925c06c8a0 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926182c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f925c0b43a0 0x7f925c14f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40644/0 (socket says 172.21.15.40:40644) 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926182c640 1 -- 172.21.15.40:0/2150174710 learned_addr learned my addr 172.21.15.40:0/2150174710 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f926102b640 1 --2- 172.21.15.40:0/2150174710 >> v2:172.21.15.204:3300/0 conn(0x7f925c06c8a0 0x7f925c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691285469 0 0) 0x7f925c0ae830 con 0x7f925c0b43a0 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9248003530 con 0x7f925c0b43a0 2024-04-02T16:12:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 328351268 0 0) 0x7f9248003530 con 0x7f925c0b43a0 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 >> v2:172.21.15.204:3300/0 conn(0x7f925c06c8a0 msgr2=0x7f925c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 --2- 172.21.15.40:0/2150174710 >> v2:172.21.15.204:3300/0 conn(0x7f925c06c8a0 0x7f925c0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 >> v1:172.21.15.204:6789/0 conn(0x7f925c0b3800 legacy=0x7f925c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0adfa0 con 0x7f925c0b43a0 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f9254001a00 con 0x7f925c0b43a0 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.556+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 >> v1:172.21.15.40:6789/0 conn(0x7f925c0b43a0 legacy=0x7f925c14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 0x7f9248003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248004810 0x7f9248004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248005220 0x7f924800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0b2b50 con 0x7f9248003a10 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f924800c170 con 0x7f9248004810 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f924800c370 con 0x7f9248005220 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 0x7f9248003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926082a640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248004810 0x7f9248004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 -- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248005220 msgr2=0x7f924800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248005220 0x7f924800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 -- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248004810 msgr2=0x7f9248004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248004810 0x7f9248004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 -- 172.21.15.40:0/2150174710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9248003530 con 0x7f9248003a10 2024-04-02T16:12:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926182c640 1 -- 172.21.15.40:0/2150174710 reap_dead start 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926102b640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 0x7f9248003e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9250009a90 tx=0x7f925001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e7407d80da8c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250009690 con 0x7f9248003a10 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92500a6410 con 0x7f9248003a10 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f924ffff640 1 -- 172.21.15.40:0/2150174710 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92500a67e0 con 0x7f9248003a10 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 -- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 msgr2=0x7f9248003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 0x7f9248003e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9250009a90 tx=0x7f925001ca30 comp rx=0 tx=0).stop 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 -- 172.21.15.40:0/2150174710 shutdown_connections 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 --2- 172.21.15.40:0/2150174710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9248003a10 0x7f9248003e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 -- 172.21.15.40:0/2150174710 >> 172.21.15.40:0/2150174710 conn(0x7f925c0b8fd0 msgr2=0x7f925c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:01.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 -- 172.21.15.40:0/2150174710 shutdown_connections 2024-04-02T16:12:01.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.560+0000 7f926328f640 1 -- 172.21.15.40:0/2150174710 wait complete. 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 Processor -- start 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- start start 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248003a10 0x7f925c123650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 0x7f925c1442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0b41a0 con 0x7f925c06cf50 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0b4020 con 0x7f925c1de2c0 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0b54d0 con 0x7f9248003a10 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57344/0 (socket says 172.21.15.40:57344) 2024-04-02T16:12:01.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 -- 172.21.15.40:0/2177444250 learned_addr learned my addr 172.21.15.40:0/2177444250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926102b640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248003a10 0x7f925c123650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926182c640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 0x7f925c1442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 -- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248003a10 msgr2=0x7f925c123650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248003a10 0x7f925c123650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 -- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 msgr2=0x7f925c1442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 0x7f925c1442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 -- 172.21.15.40:0/2177444250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c123ea0 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926182c640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 0x7f925c1442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926082a640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f925800d4f0 tx=0x7f925800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258005480 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9258017070 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c06dc70 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258003820 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c124160 con 0x7f925c06cf50 2024-04-02T16:12:01.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c0685f0 con 0x7f925c06cf50 2024-04-02T16:12:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f92580039c0 con 0x7f925c06cf50 2024-04-02T16:12:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 0x7f924006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f92580866d0 con 0x7f925c06cf50 2024-04-02T16:12:01.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.564+0000 7f926102b640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 0x7f924006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:01.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.568+0000 7f926102b640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 0x7f924006a870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92500ad000 tx=0x7f92500a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:01.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.568+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f92580566d0 con 0x7f925c06cf50 2024-04-02T16:12:01.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:12:01.714 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:12:01.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f925c0b7fc0 con 0x7f925c06cf50 2024-04-02T16:12:01.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f924dffb640 1 -- 172.21.15.40:0/2177444250 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f925805a9d0 con 0x7f925c06cf50 2024-04-02T16:12:01.737 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 msgr2=0x7f924006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 0x7f924006a870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f92500ad000 tx=0x7f92500a9040 comp rx=0 tx=0).stop 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 msgr2=0x7f925c1ddd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f925800d4f0 tx=0x7f925800d9b0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 shutdown_connections 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.736+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f92400683b0 0x7f924006a870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f925c1de2c0 0x7f925c1442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f925c06cf50 0x7f925c1ddd80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 --2- 172.21.15.40:0/2177444250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9248003a10 0x7f925c123650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:01.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 >> 172.21.15.40:0/2177444250 conn(0x7f925c0b8fd0 msgr2=0x7f925c0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:01.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 shutdown_connections 2024-04-02T16:12:01.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:01.740+0000 7f926328f640 1 -- 172.21.15.40:0/2177444250 wait complete. 2024-04-02T16:12:01.755 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-04-02T16:12:01.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:01.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:01.830 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:12:01.831 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:12:01.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:01.831 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T16:12:01.891 INFO:tasks.cephfs.cephfs_test_case:[] 2024-04-02T16:12:01.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:02.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 Processor -- start 2024-04-02T16:12:02.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 -- start start 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ff40b4060 0x7f2ff4145950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff40b5610 con 0x7f2ff40b47c0 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff40b6450 con 0x7f2ff40b6070 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffc4ec640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff4073330 con 0x7f2ff40b4060 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffaa89640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ff40b47c0 0x7f2ff414fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40652/0 (socket says 172.21.15.40:40652) 2024-04-02T16:12:02.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffaa89640 1 -- 172.21.15.40:0/3587526394 learned_addr learned my addr 172.21.15.40:0/3587526394 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9a87640 1 --2- 172.21.15.40:0/3587526394 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40b4060 0x7f2ff4145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1291988486 0 0) 0x7f2ff40b5610 con 0x7f2ff40b47c0 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fe0003530 con 0x7f2ff40b47c0 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2796287825 0 0) 0x7f2fe0003530 con 0x7f2ff40b47c0 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40b4060 msgr2=0x7f2ff4145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 --2- 172.21.15.40:0/3587526394 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40b4060 0x7f2ff4145950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 >> v1:172.21.15.204:6789/0 conn(0x7f2ff40b6070 legacy=0x7f2ff40b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff4072d70 con 0x7f2ff40b47c0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f2fec001a00 con 0x7f2ff40b47c0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 >> v1:172.21.15.40:6789/0 conn(0x7f2ff40b47c0 legacy=0x7f2ff414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2fe0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe0004810 0x7f2fe0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe00053d0 0x7f2fe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4073330 con 0x7f2fe00039c0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe000c270 con 0x7f2fe00053d0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe000c470 con 0x7f2fe0004810 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2fe0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffaa89640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe00053d0 0x7f2fe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 -- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe0004810 msgr2=0x7f2fe0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fe0004810 0x7f2fe0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 -- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe00053d0 msgr2=0x7f2fe000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe00053d0 0x7f2fe000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffa288640 1 -- 172.21.15.40:0/3587526394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe0003530 con 0x7f2fe00039c0 2024-04-02T16:12:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.064+0000 7f2ffaa89640 1 -- 172.21.15.40:0/3587526394 reap_dead start 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2fe0003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0001780 tx=0x7f2ff001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0279685ad7b9b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00013a0 con 0x7f2fe00039c0 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff00052f0 con 0x7f2fe00039c0 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ff9286640 1 -- 172.21.15.40:0/3587526394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0003840 con 0x7f2fe00039c0 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 msgr2=0x7f2fe0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2fe0003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0001780 tx=0x7f2ff001c400 comp rx=0 tx=0).stop 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3587526394 shutdown_connections 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3587526394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2fe0003e00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3587526394 >> 172.21.15.40:0/3587526394 conn(0x7f2ff406e600 msgr2=0x7f2ff406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3587526394 shutdown_connections 2024-04-02T16:12:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3587526394 wait complete. 2024-04-02T16:12:02.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 Processor -- start 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.068+0000 7f2ffc4ec640 1 -- start start 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff413ce80 0x7f2ff4141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff41429f0 0x7f2ff41e1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff40b52e0 con 0x7f2fe00039c0 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff40b5160 con 0x7f2ff41429f0 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff406b580 con 0x7f2ff413ce80 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57354/0 (socket says 172.21.15.40:57354) 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 -- 172.21.15.40:0/3719163717 learned_addr learned my addr 172.21.15.40:0/3719163717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 -- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff413ce80 msgr2=0x7f2ff4141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffaa89640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff41429f0 0x7f2ff41e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ff9a87640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff413ce80 0x7f2ff4141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff413ce80 0x7f2ff4141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 -- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff41429f0 msgr2=0x7f2ff41e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff41429f0 0x7f2ff41e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 -- 172.21.15.40:0/3719163717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff41406c0 con 0x7f2fe00039c0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffa288640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ff001c230 tx=0x7f2fe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0007a40 con 0x7f2fe00039c0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff00ba070 con 0x7f2fe00039c0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff4140960 con 0x7f2fe00039c0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00054a0 con 0x7f2fe00039c0 2024-04-02T16:12:02.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff4142e70 con 0x7f2fe00039c0 2024-04-02T16:12:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4068d00 con 0x7f2fe00039c0 2024-04-02T16:12:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2ff0007030 con 0x7f2fe00039c0 2024-04-02T16:12:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 0x7f2fd806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ff0120950 con 0x7f2fe00039c0 2024-04-02T16:12:02.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.072+0000 7f2ff9a87640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 0x7f2fd806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.076+0000 7f2ff9a87640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 0x7f2fd806a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4006aa0 tx=0x7f2fe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.080+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2ff0016110 con 0x7f2fe00039c0 2024-04-02T16:12:02.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.276+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ff40b7f40 con 0x7f2fe00039c0 2024-04-02T16:12:02.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.276+0000 7f2feaffd640 1 -- 172.21.15.40:0/3719163717 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7f2ff00f0fa0 con 0x7f2fe00039c0 2024-04-02T16:12:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:02.278 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:12:02.278 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 msgr2=0x7f2fd806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 0x7f2fd806a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4006aa0 tx=0x7f2fe4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 msgr2=0x7f2ff4141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ff001c230 tx=0x7f2fe0005840 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 shutdown_connections 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff413ce80 0x7f2ff4141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2fd8068380 0x7f2fd806a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff41429f0 0x7f2ff41e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 --2- 172.21.15.40:0/3719163717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe00039c0 0x7f2ff4141e20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 >> 172.21.15.40:0/3719163717 conn(0x7f2ff406e600 msgr2=0x7f2ff40b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 shutdown_connections 2024-04-02T16:12:02.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.280+0000 7f2ffc4ec640 1 -- 172.21.15.40:0/3719163717 wait complete. 2024-04-02T16:12:02.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:02.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 Processor -- start 2024-04-02T16:12:02.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 -- start start 2024-04-02T16:12:02.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c3c147c50 0x7f1c3c1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1c3c148620 0x7f1c3c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3c148a90 con 0x7f1c3c06c8a0 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3c0b2b50 con 0x7f1c3c147c50 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.420+0000 7f1c4149d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1c3c0b0300 con 0x7f1c3c148620 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c3c147c50 0x7f1c3c1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c3c147c50 0x7f1c3c1480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57370/0 (socket says 172.21.15.40:57370) 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 -- 172.21.15.40:0/2628156243 learned_addr learned my addr 172.21.15.40:0/2628156243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3b7fe640 1 --2- 172.21.15.40:0/2628156243 >> v2:172.21.15.204:3300/0 conn(0x7f1c3c148620 0x7f1c3c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 -- 172.21.15.40:0/2628156243 >> v2:172.21.15.204:3300/0 conn(0x7f1c3c148620 msgr2=0x7f1c3c14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 --2- 172.21.15.40:0/2628156243 >> v2:172.21.15.204:3300/0 conn(0x7f1c3c148620 0x7f1c3c14f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 -- 172.21.15.40:0/2628156243 >> v1:172.21.15.204:6789/0 conn(0x7f1c3c06c8a0 legacy=0x7f1c3c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 -- 172.21.15.40:0/2628156243 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c0ae830 con 0x7f1c3c147c50 2024-04-02T16:12:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c3a7fc640 1 --2- 172.21.15.40:0/2628156243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c147c50 0x7f1c3c1480e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c072f90 tx=0x7f1c3001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316519ef037f820d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c39ffb640 1 -- 172.21.15.40:0/2628156243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c300b0070 con 0x7f1c3c147c50 2024-04-02T16:12:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c39ffb640 1 -- 172.21.15.40:0/2628156243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c300ab600 con 0x7f1c3c147c50 2024-04-02T16:12:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c39ffb640 1 -- 172.21.15.40:0/2628156243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c300ab9d0 con 0x7f1c3c147c50 2024-04-02T16:12:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 -- 172.21.15.40:0/2628156243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c147c50 msgr2=0x7f1c3c1480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2628156243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c147c50 0x7f1c3c1480e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c072f90 tx=0x7f1c3001c490 comp rx=0 tx=0).stop 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 -- 172.21.15.40:0/2628156243 shutdown_connections 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2628156243 >> v2:172.21.15.204:3300/0 conn(0x7f1c3c148620 0x7f1c3c14f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2628156243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c147c50 0x7f1c3c1480e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 -- 172.21.15.40:0/2628156243 >> 172.21.15.40:0/2628156243 conn(0x7f1c3c0b8fd0 msgr2=0x7f1c3c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 -- 172.21.15.40:0/2628156243 shutdown_connections 2024-04-02T16:12:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.424+0000 7f1c4149d640 1 -- 172.21.15.40:0/2628156243 wait complete. 2024-04-02T16:12:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 Processor -- start 2024-04-02T16:12:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- start start 2024-04-02T16:12:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 0x7f1c3c0b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c147c50 0x7f1c3c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c0b3130 con 0x7f1c3c148620 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c0b0540 con 0x7f1c3c147c50 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c0726e0 con 0x7f1c3c06c8a0 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57380/0 (socket says 172.21.15.40:57380) 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 -- 172.21.15.40:0/2598919277 learned_addr learned my addr 172.21.15.40:0/2598919277 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3a7fc640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c147c50 0x7f1c3c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3affd640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 0x7f1c3c0b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 -- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 msgr2=0x7f1c3c0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 0x7f1c3c0b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 -- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c147c50 msgr2=0x7f1c3c06f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c147c50 0x7f1c3c06f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 -- 172.21.15.40:0/2598919277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c070750 con 0x7f1c3c148620 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3affd640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 0x7f1c3c0b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c3b7fe640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007e40 tx=0x7f1c3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34014620 con 0x7f1c3c148620 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34019070 con 0x7f1c3c148620 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34014a20 con 0x7f1c3c148620 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c072250 con 0x7f1c3c148620 2024-04-02T16:12:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c0b8740 con 0x7f1c3c148620 2024-04-02T16:12:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.428+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c3c0685f0 con 0x7f1c3c148620 2024-04-02T16:12:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.432+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1c34014bc0 con 0x7f1c3c148620 2024-04-02T16:12:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.432+0000 7f1c237fe640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 0x7f1c2806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.432+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1c34087230 con 0x7f1c3c148620 2024-04-02T16:12:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.432+0000 7f1c3affd640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 0x7f1c2806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.432+0000 7f1c3affd640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 0x7f1c2806a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c001670 tx=0x7f1c2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.436+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f1c340577e0 con 0x7f1c3c148620 2024-04-02T16:12:02.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.600+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c3c0b32b0 con 0x7f1c3c148620 2024-04-02T16:12:02.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.600+0000 7f1c237fe640 1 -- 172.21.15.40:0/2598919277 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f1c340570e0 con 0x7f1c3c148620 2024-04-02T16:12:02.604 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:02.604 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 msgr2=0x7f1c2806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 0x7f1c2806a840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c001670 tx=0x7f1c2c009040 comp rx=0 tx=0).stop 2024-04-02T16:12:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 msgr2=0x7f1c3c13ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007e40 tx=0x7f1c3400e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 shutdown_connections 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1c28068380 0x7f1c2806a840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c3c148620 0x7f1c3c13ff20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c3c147c50 0x7f1c3c06f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 --2- 172.21.15.40:0/2598919277 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c3c06c8a0 0x7f1c3c0b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 >> 172.21.15.40:0/2598919277 conn(0x7f1c3c0b8fd0 msgr2=0x7f1c3c06ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 shutdown_connections 2024-04-02T16:12:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.604+0000 7f1c4149d640 1 -- 172.21.15.40:0/2598919277 wait complete. 2024-04-02T16:12:02.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:11:29.810061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:11:30.438845+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:12:02.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:12:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.752+0000 7f35dbefc640 1 Processor -- start 2024-04-02T16:12:02.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.752+0000 7f35dbefc640 1 -- start start 2024-04-02T16:12:02.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35dbefc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f35d409f0e0 0x7f35d40713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35dbefc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d4071900 con 0x7f35d40b0550 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35dbefc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d409f4c0 con 0x7f35d4070910 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35dbefc640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f35d4072f20 con 0x7f35d409f0e0 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f35d40b0550 0x7f35d4152060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40654/0 (socket says 172.21.15.40:40654) 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 -- 172.21.15.40:0/3902955284 learned_addr learned my addr 172.21.15.40:0/3902955284 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d9c98640 1 --2- 172.21.15.40:0/3902955284 >> v2:172.21.15.204:3300/0 conn(0x7f35d409f0e0 0x7f35d40713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675956491 0 0) 0x7f35d4071900 con 0x7f35d40b0550 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35bc003530 con 0x7f35d40b0550 2024-04-02T16:12:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551520628 0 0) 0x7f35bc003530 con 0x7f35d40b0550 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 >> v2:172.21.15.204:3300/0 conn(0x7f35d409f0e0 msgr2=0x7f35d40713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 --2- 172.21.15.40:0/3902955284 >> v2:172.21.15.204:3300/0 conn(0x7f35d409f0e0 0x7f35d40713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 >> v1:172.21.15.204:6789/0 conn(0x7f35d4070910 legacy=0x7f35d4141660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d40b0ff0 con 0x7f35d40b0550 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f35c4001680 con 0x7f35d40b0550 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 >> v1:172.21.15.40:6789/0 conn(0x7f35d40b0550 legacy=0x7f35d4152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35bc0039c0 0x7f35bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc0043c0 0x7f35bc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 0x7f35bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc00bfc0 con 0x7f35bc005090 2024-04-02T16:12:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc00c170 con 0x7f35bc0039c0 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f35bc00c370 con 0x7f35bc0043c0 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 0x7f35bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d9497640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc0043c0 0x7f35bc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35d9c98640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35bc0039c0 0x7f35bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 -- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc0043c0 msgr2=0x7f35bc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc0043c0 0x7f35bc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 -- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35bc0039c0 msgr2=0x7f35bc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35bc0039c0 0x7f35bc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.756+0000 7f35da499640 1 -- 172.21.15.40:0/3902955284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc003530 con 0x7f35bc005090 2024-04-02T16:12:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35da499640 1 -- 172.21.15.40:0/3902955284 reap_dead start 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35da499640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 0x7f35bc00b9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c2c0 tx=0x7f35c401f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1402dd1fbffba16e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4007d60 con 0x7f35bc005090 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4005b00 con 0x7f35bc005090 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d8c96640 1 -- 172.21.15.40:0/3902955284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4007070 con 0x7f35bc005090 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 msgr2=0x7f35bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 0x7f35bc00b9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c2c0 tx=0x7f35c401f490 comp rx=0 tx=0).stop 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- 172.21.15.40:0/3902955284 shutdown_connections 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3902955284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc005090 0x7f35bc00b9d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- 172.21.15.40:0/3902955284 >> 172.21.15.40:0/3902955284 conn(0x7f35d40b7f50 msgr2=0x7f35d40b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- 172.21.15.40:0/3902955284 shutdown_connections 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- 172.21.15.40:0/3902955284 wait complete. 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 Processor -- start 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- start start 2024-04-02T16:12:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc005090 0x7f35d413e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d4142a10 0x7f35d4142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40b3a70 con 0x7f35d4144320 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35d40734c0 con 0x7f35d4142a10 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35dbefc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f35d40b3bf0 con 0x7f35bc005090 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57400/0 (socket says 172.21.15.40:57400) 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 -- 172.21.15.40:0/3415336858 learned_addr learned my addr 172.21.15.40:0/3415336858 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9c98640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc005090 0x7f35d413e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35da499640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d4142a10 0x7f35d4142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 -- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc005090 msgr2=0x7f35d413e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc005090 0x7f35d413e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 -- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d4142a10 msgr2=0x7f35d4142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d4142a10 0x7f35d4142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 -- 172.21.15.40:0/3415336858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d413cd00 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.760+0000 7f35d9497640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e960 tx=0x7f35c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8003b90 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c8018070 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d4143b20 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80050d0 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d413df10 con 0x7f35d4144320 2024-04-02T16:12:02.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4068d00 con 0x7f35d4144320 2024-04-02T16:12:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f35c8005270 con 0x7f35d4144320 2024-04-02T16:12:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 0x7f35c006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35c80869c0 con 0x7f35d4144320 2024-04-02T16:12:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d9c98640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 0x7f35c006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.764+0000 7f35d9c98640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 0x7f35c006a870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35d40b6030 tx=0x7f35cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.768+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f35c8088e20 con 0x7f35d4144320 2024-04-02T16:12:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.932+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35d409f4c0 con 0x7f35d4144320 2024-04-02T16:12:02.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.932+0000 7f35d27fc640 1 -- 172.21.15.40:0/3415336858 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f35c801a310 con 0x7f35d4144320 2024-04-02T16:12:02.934 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 msgr2=0x7f35c006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 0x7f35c006a870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35d40b6030 tx=0x7f35cc008040 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 msgr2=0x7f35d41dcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e960 tx=0x7f35c800ee20 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 shutdown_connections 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f35c00683b0 0x7f35c006a870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d4142a10 0x7f35d4142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35d4144320 0x7f35d41dcbd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 --2- 172.21.15.40:0/3415336858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35bc005090 0x7f35d413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 >> 172.21.15.40:0/3415336858 conn(0x7f35d40b7f50 msgr2=0x7f35d40672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 shutdown_connections 2024-04-02T16:12:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:02.936+0000 7f35dbefc640 1 -- 172.21.15.40:0/3415336858 wait complete. 2024-04-02T16:12:02.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:12:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 Processor -- start 2024-04-02T16:12:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 -- start start 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc2cc073230 0x7fc2cc0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0b19e0 con 0x7fc2cc148e50 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc0b05c0 con 0x7fc2cc0b23d0 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d2a3f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0b08a0 con 0x7fc2cc073230 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d0fdc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc2cc148e50 0x7fc2cc149280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40666/0 (socket says 172.21.15.40:40666) 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1612052392 learned_addr learned my addr 172.21.15.40:0/1612052392 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1612052392 >> v2:172.21.15.204:3300/0 conn(0x7fc2cc073230 0x7fc2cc0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19207305 0 0) 0x7fc2cc0b19e0 con 0x7fc2cc148e50 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2b4003530 con 0x7fc2cc148e50 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1251934582 0 0) 0x7fc2b4003530 con 0x7fc2cc148e50 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 >> v2:172.21.15.204:3300/0 conn(0x7fc2cc073230 msgr2=0x7fc2cc0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 --2- 172.21.15.40:0/1612052392 >> v2:172.21.15.204:3300/0 conn(0x7fc2cc073230 0x7fc2cc0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 >> v1:172.21.15.204:6789/0 conn(0x7fc2cc0b23d0 legacy=0x7fc2cc09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0b0b80 con 0x7fc2cc148e50 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fc2c4001a00 con 0x7fc2cc148e50 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 >> v1:172.21.15.40:6789/0 conn(0x7fc2cc148e50 legacy=0x7fc2cc149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2b40039c0 0x7fc2b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 0x7fc2b4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b4005130 0x7fc2b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b08a0 con 0x7fc2b40043c0 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b400c070 con 0x7fc2b4005130 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b400c270 con 0x7fc2b40039c0 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 0x7fc2b4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d0fdc640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b4005130 0x7fc2b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2b40039c0 0x7fc2b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2b40039c0 msgr2=0x7fc2b4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2b40039c0 0x7fc2b4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b4005130 msgr2=0x7fc2b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b4005130 0x7fc2b400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1612052392 reap_dead start 2024-04-02T16:12:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.084+0000 7fc2cb7fe640 1 -- 172.21.15.40:0/1612052392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b4003530 con 0x7fc2b40043c0 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 0x7fc2b4004890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009ae0 tx=0x7fc2c001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1ec95f95f4d3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00b2070 con 0x7fc2b40043c0 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c00a5660 con 0x7fc2b40043c0 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2caffd640 1 -- 172.21.15.40:0/1612052392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00096b0 con 0x7fc2b40043c0 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 msgr2=0x7fc2b4004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 0x7fc2b4004890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009ae0 tx=0x7fc2c001ca70 comp rx=0 tx=0).stop 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1612052392 shutdown_connections 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1612052392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2b40043c0 0x7fc2b4004890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1612052392 >> 172.21.15.40:0/1612052392 conn(0x7fc2cc0b8fd0 msgr2=0x7fc2cc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1612052392 shutdown_connections 2024-04-02T16:12:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1612052392 wait complete. 2024-04-02T16:12:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 Processor -- start 2024-04-02T16:12:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- start start 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 0x7fc2cc141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 0x7fc2cc1e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b2930 con 0x7fc2cc146bb0 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b27b0 con 0x7fc2b40043c0 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d2a3f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0b2ab0 con 0x7fc2cc1e0a80 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.088+0000 7fc2d0fdc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57422/0 (socket says 172.21.15.40:57422) 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1590803200 learned_addr learned my addr 172.21.15.40:0/1590803200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 0x7fc2cc1e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 0x7fc2cc141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 msgr2=0x7fc2cc1e0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 0x7fc2cc1e0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 msgr2=0x7fc2cc141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 0x7fc2cc141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 -- 172.21.15.40:0/1590803200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc140f30 con 0x7fc2cc146bb0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 0x7fc2cc141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cb7fe640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 0x7fc2cc1e0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d0fdc640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0b3cb0 tx=0x7fc2c4007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400aad0 con 0x7fc2cc146bb0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c4019070 con 0x7fc2cc146bb0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40133f0 con 0x7fc2cc146bb0 2024-04-02T16:12:03.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc141ee0 con 0x7fc2cc146bb0 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1466b0 con 0x7fc2cc146bb0 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc0685f0 con 0x7fc2cc146bb0 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2c4013590 con 0x7fc2cc146bb0 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 0x7fc2b806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc2c4086e60 con 0x7fc2cc146bb0 2024-04-02T16:12:03.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 0x7fc2b806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.092+0000 7fc2cbfff640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 0x7fc2b806a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005630 tx=0x7fc2bc006ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.096+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc2c4057410 con 0x7fc2cc146bb0 2024-04-02T16:12:03.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.260+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc2cc0b1690 con 0x7fc2cc146bb0 2024-04-02T16:12:03.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.260+0000 7fc2c8ff9640 1 -- 172.21.15.40:0/1590803200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc2c4056d10 con 0x7fc2cc146bb0 2024-04-02T16:12:03.263 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:12:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 msgr2=0x7fc2b806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 0x7fc2b806a840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005630 tx=0x7fc2bc006ff0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 msgr2=0x7fc2cc147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0b3cb0 tx=0x7fc2c4007b00 comp rx=0 tx=0).stop 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 shutdown_connections 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2cc1e0a80 0x7fc2cc1e0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc2b8068380 0x7fc2b806a840 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2cc146bb0 0x7fc2cc147030 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 --2- 172.21.15.40:0/1590803200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2b40043c0 0x7fc2cc141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 >> 172.21.15.40:0/1590803200 conn(0x7fc2cc0b8fd0 msgr2=0x7fc2cc06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 shutdown_connections 2024-04-02T16:12:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.264+0000 7fc2d2a3f640 1 -- 172.21.15.40:0/1590803200 wait complete. 2024-04-02T16:12:03.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:12:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 Processor -- start 2024-04-02T16:12:03.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 -- start start 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3fbc0b1c10 0x7f3fbc0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fbc0b0a10 con 0x7f3fbc0b25b0 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fbc0b3210 con 0x7f3fbc0ae080 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc3027640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3fbc0b2a00 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fbbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3fbc0b1c10 0x7f3fbc0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc15c4640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3fbc0b25b0 0x7f3fbc1522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40672/0 (socket says 172.21.15.40:40672) 2024-04-02T16:12:03.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fc15c4640 1 -- 172.21.15.40:0/2979680563 learned_addr learned my addr 172.21.15.40:0/2979680563 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fbbfff640 1 -- 172.21.15.40:0/2979680563 >> v1:172.21.15.204:6789/0 conn(0x7f3fbc0ae080 legacy=0x7f3fbc0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fbbfff640 1 -- 172.21.15.40:0/2979680563 >> v1:172.21.15.40:6789/0 conn(0x7f3fbc0b25b0 legacy=0x7f3fbc1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.412+0000 7f3fbbfff640 1 -- 172.21.15.40:0/2979680563 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc06db10 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 --2- 172.21.15.40:0/2979680563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc0b2070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0b2b80 tx=0x7f3fb001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68110754cbeb174 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbb7fe640 1 -- 172.21.15.40:0/2979680563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00b1070 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbb7fe640 1 -- 172.21.15.40:0/2979680563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00ac5f0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbb7fe640 1 -- 172.21.15.40:0/2979680563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00ac9c0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- 172.21.15.40:0/2979680563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 msgr2=0x7f3fbc0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 --2- 172.21.15.40:0/2979680563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc0b2070 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0b2b80 tx=0x7f3fb001c4a0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- 172.21.15.40:0/2979680563 shutdown_connections 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 --2- 172.21.15.40:0/2979680563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc0b2070 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- 172.21.15.40:0/2979680563 >> 172.21.15.40:0/2979680563 conn(0x7f3fbc0b8fd0 msgr2=0x7f3fbc0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- 172.21.15.40:0/2979680563 shutdown_connections 2024-04-02T16:12:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- 172.21.15.40:0/2979680563 wait complete. 2024-04-02T16:12:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 Processor -- start 2024-04-02T16:12:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- start start 2024-04-02T16:12:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fbc0ae080 0x7f3fbc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fbc0b25b0 0x7f3fbc09b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc0ae760 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc0afe60 con 0x7f3fbc0ae080 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc3027640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc0af2f0 con 0x7f3fbc0b25b0 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57450/0 (socket says 172.21.15.40:57450) 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 -- 172.21.15.40:0/1142964802 learned_addr learned my addr 172.21.15.40:0/1142964802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 -- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fbc0b25b0 msgr2=0x7f3fbc09b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc15c4640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fbc0b25b0 0x7f3fbc09b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fc0dc3640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fbc0ae080 0x7f3fbc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fbc0b25b0 0x7f3fbc09b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 -- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fbc0ae080 msgr2=0x7f3fbc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fbc0ae080 0x7f3fbc0721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.416+0000 7f3fbbfff640 1 -- 172.21.15.40:0/1142964802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc09c2f0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fbbfff640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00af040 tx=0x7f3fb0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00b9070 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00b5610 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc1e3ff0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00b1040 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc09bbb0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fbc09c5d0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3fb00bc070 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 0x7f3fa806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3fb0120900 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fc0dc3640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 0x7f3fa806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.420+0000 7f3fc0dc3640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 0x7f3fa806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3fac007fd0 tx=0x7f3fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.424+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3fbc09c5d0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fbc0685f0 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fb97fa640 1 -- 172.21.15.40:0/1142964802 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3fb00a9070 con 0x7f3fbc0b1c10 2024-04-02T16:12:03.589 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 msgr2=0x7f3fa806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 0x7f3fa806a840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3fac007fd0 tx=0x7f3fac009040 comp rx=0 tx=0).stop 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 msgr2=0x7f3fbc1e3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00af040 tx=0x7f3fb0019240 comp rx=0 tx=0).stop 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 shutdown_connections 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fbc0b25b0 0x7f3fbc09b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3fa8068380 0x7f3fa806a840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0b1c10 0x7f3fbc1e3ab0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 --2- 172.21.15.40:0/1142964802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fbc0ae080 0x7f3fbc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 >> 172.21.15.40:0/1142964802 conn(0x7f3fbc0b8fd0 msgr2=0x7f3fbc0b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 shutdown_connections 2024-04-02T16:12:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.588+0000 7f3fc3027640 1 -- 172.21.15.40:0/1142964802 wait complete. 2024-04-02T16:12:03.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 Processor -- start 2024-04-02T16:12:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 -- start start 2024-04-02T16:12:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef40b4bf0 0x7fbef40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b8c80 0x7fbef40b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef40b6310 con 0x7fbef40b3500 2024-04-02T16:12:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbef40b55d0 con 0x7fbef40b4bf0 2024-04-02T16:12:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbefb60b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbef414b560 con 0x7fbef40b8c80 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef40b4bf0 0x7fbef40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbef40b4bf0 0x7fbef40b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57454/0 (socket says 172.21.15.40:57454) 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 -- 172.21.15.40:0/995812130 learned_addr learned my addr 172.21.15.40:0/995812130 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef9ba8640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b8c80 0x7fbef40b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 -- 172.21.15.40:0/995812130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b8c80 msgr2=0x7fbef40b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b8c80 0x7fbef40b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 -- 172.21.15.40:0/995812130 >> v1:172.21.15.204:6789/0 conn(0x7fbef40b3500 legacy=0x7fbef414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.736+0000 7fbef93a7640 1 -- 172.21.15.40:0/995812130 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef40b3fa0 con 0x7fbef40b4bf0 2024-04-02T16:12:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b4bf0 0x7fbef40b4fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009870 tx=0x7fbef001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c238bdcf2d8b228c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbeebfff640 1 -- 172.21.15.40:0/995812130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00b0070 con 0x7fbef40b4bf0 2024-04-02T16:12:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbeebfff640 1 -- 172.21.15.40:0/995812130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef00ab600 con 0x7fbef40b4bf0 2024-04-02T16:12:03.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbeebfff640 1 -- 172.21.15.40:0/995812130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00ab9d0 con 0x7fbef40b4bf0 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- 172.21.15.40:0/995812130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b4bf0 msgr2=0x7fbef40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b4bf0 0x7fbef40b4fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009870 tx=0x7fbef001c440 comp rx=0 tx=0).stop 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- 172.21.15.40:0/995812130 shutdown_connections 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b8c80 0x7fbef40b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- 172.21.15.40:0/995812130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b4bf0 0x7fbef40b4fd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- 172.21.15.40:0/995812130 >> 172.21.15.40:0/995812130 conn(0x7fbef40734b0 msgr2=0x7fbef40bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- 172.21.15.40:0/995812130 shutdown_connections 2024-04-02T16:12:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- 172.21.15.40:0/995812130 wait complete. 2024-04-02T16:12:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 Processor -- start 2024-04-02T16:12:03.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- start start 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b4bf0 0x7fbef41dff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef40b8c80 0x7fbef4140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4154c10 con 0x7fbef40b3500 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4154d90 con 0x7fbef40b8c80 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbefb60b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbef4154f10 con 0x7fbef40b4bf0 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57456/0 (socket says 172.21.15.40:57456) 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 -- 172.21.15.40:0/3770761820 learned_addr learned my addr 172.21.15.40:0/3770761820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef9ba8640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef40b8c80 0x7fbef4140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef8ba6640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b4bf0 0x7fbef41dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 -- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b4bf0 msgr2=0x7fbef41dff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b4bf0 0x7fbef41dff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 -- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef40b8c80 msgr2=0x7fbef4140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef40b8c80 0x7fbef4140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.740+0000 7fbef93a7640 1 -- 172.21.15.40:0/3770761820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef40bd340 con 0x7fbef40b3500 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbef93a7640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ae040 tx=0x7fbef001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00ba020 con 0x7fbef40b3500 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef00aa9f0 con 0x7fbef40b3500 2024-04-02T16:12:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef413f350 con 0x7fbef40b3500 2024-04-02T16:12:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00b0040 con 0x7fbef40b3500 2024-04-02T16:12:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef4144f90 con 0x7fbef40b3500 2024-04-02T16:12:03.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef41e39c0 con 0x7fbef40b3500 2024-04-02T16:12:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbef00bd020 con 0x7fbef40b3500 2024-04-02T16:12:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 0x7fbed806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbef0120ff0 con 0x7fbef40b3500 2024-04-02T16:12:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbef8ba6640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 0x7fbed806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:03.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.744+0000 7fbef8ba6640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 0x7fbed806a870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbee4007fd0 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:03.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.748+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fbef00c4030 con 0x7fbef40b3500 2024-04-02T16:12:03.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef41451c0 con 0x7fbef40b3500 2024-04-02T16:12:03.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbee9ffb640 1 -- 172.21.15.40:0/3770761820 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7fbef0016150 con 0x7fbef40b3500 2024-04-02T16:12:03.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:03.945 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:12:03.945 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 msgr2=0x7fbed806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 0x7fbed806a870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbee4007fd0 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 msgr2=0x7fbef4147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbef00ae040 tx=0x7fbef001ac80 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 shutdown_connections 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef40b4bf0 0x7fbef41dff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbed80683b0 0x7fbed806a870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef40b8c80 0x7fbef4140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 --2- 172.21.15.40:0/3770761820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbef40b3500 0x7fbef4147600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:03.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 >> 172.21.15.40:0/3770761820 conn(0x7fbef40734b0 msgr2=0x7fbef40bc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:03.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 shutdown_connections 2024-04-02T16:12:03.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:03.944+0000 7fbefb60b640 1 -- 172.21.15.40:0/3770761820 wait complete. 2024-04-02T16:12:03.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 Processor -- start 2024-04-02T16:12:04.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 -- start start 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7e4147c50 0x7fb7e41480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e4148a90 con 0x7fb7e4148620 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e40ae830 con 0x7fb7e406c8a0 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7ea99a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e40b2b50 con 0x7fb7e4147c50 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb7e4147c50 0x7fb7e41480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e8f37640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7e4148620 0x7fb7e414f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40680/0 (socket says 172.21.15.40:40680) 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e8f37640 1 -- 172.21.15.40:0/995485124 learned_addr learned my addr 172.21.15.40:0/995485124 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e37fe640 1 -- 172.21.15.40:0/995485124 >> v1:172.21.15.204:6789/0 conn(0x7fb7e406c8a0 legacy=0x7fb7e40af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e37fe640 1 -- 172.21.15.40:0/995485124 >> v1:172.21.15.40:6789/0 conn(0x7fb7e4148620 legacy=0x7fb7e414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e37fe640 1 -- 172.21.15.40:0/995485124 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e406cc80 con 0x7fb7e4147c50 2024-04-02T16:12:04.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.072+0000 7fb7e2ffd640 1 -- 172.21.15.40:0/995485124 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429864204 0 0) 0x7fb7e4148a90 con 0x7fb7e4148620 2024-04-02T16:12:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7e37fe640 1 --2- 172.21.15.40:0/995485124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e4147c50 0x7fb7e41480e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b2cd0 tx=0x7fb7d801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8018ad82a7ee3e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7e2ffd640 1 -- 172.21.15.40:0/995485124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80b0070 con 0x7fb7e4147c50 2024-04-02T16:12:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7e2ffd640 1 -- 172.21.15.40:0/995485124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80ab600 con 0x7fb7e4147c50 2024-04-02T16:12:04.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7e2ffd640 1 -- 172.21.15.40:0/995485124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80a4e60 con 0x7fb7e4147c50 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- 172.21.15.40:0/995485124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e4147c50 msgr2=0x7fb7e41480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/995485124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e4147c50 0x7fb7e41480e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b2cd0 tx=0x7fb7d801c3d0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- 172.21.15.40:0/995485124 shutdown_connections 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/995485124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e4147c50 0x7fb7e41480e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- 172.21.15.40:0/995485124 >> 172.21.15.40:0/995485124 conn(0x7fb7e40b8fd0 msgr2=0x7fb7e40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- 172.21.15.40:0/995485124 shutdown_connections 2024-04-02T16:12:04.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- 172.21.15.40:0/995485124 wait complete. 2024-04-02T16:12:04.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 Processor -- start 2024-04-02T16:12:04.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.076+0000 7fb7ea99a640 1 -- start start 2024-04-02T16:12:04.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7e4147c50 0x7fb7e4149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7e4148620 0x7fb7e4146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40b3130 con 0x7fb7e406c8a0 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40b08c0 con 0x7fb7e4148620 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e4146090 con 0x7fb7e4147c50 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57488/0 (socket says 172.21.15.40:57488) 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 -- 172.21.15.40:0/653542374 learned_addr learned my addr 172.21.15.40:0/653542374 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e37fe640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7e4147c50 0x7fb7e4149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 -- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7e4147c50 msgr2=0x7fb7e4149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7e4147c50 0x7fb7e4149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 -- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7e4148620 msgr2=0x7fb7e4146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7e4148620 0x7fb7e4146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 -- 172.21.15.40:0/653542374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e414a620 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e3fff640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c7c0 tx=0x7fb7dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc016020 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00aa50 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e414a900 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc012420 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e4149410 con 0x7fb7e406c8a0 2024-04-02T16:12:04.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e40685f0 con 0x7fb7e406c8a0 2024-04-02T16:12:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7dc019020 con 0x7fb7e406c8a0 2024-04-02T16:12:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 0x7fb7d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7dc086a70 con 0x7fb7e406c8a0 2024-04-02T16:12:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.080+0000 7fb7e37fe640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 0x7fb7d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.084+0000 7fb7e37fe640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 0x7fb7d006a870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80ad000 tx=0x7fb7d80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.084+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb7dc020030 con 0x7fb7e406c8a0 2024-04-02T16:12:04.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.244+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7e40b76b0 con 0x7fb7e406c8a0 2024-04-02T16:12:04.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7e0ff9640 1 -- 172.21.15.40:0/653542374 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7fb7dc057020 con 0x7fb7e406c8a0 2024-04-02T16:12:04.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:04.248 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 msgr2=0x7fb7d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 0x7fb7d006a870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80ad000 tx=0x7fb7d80a6000 comp rx=0 tx=0).stop 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 msgr2=0x7fb7e4072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c7c0 tx=0x7fb7dc00cc80 comp rx=0 tx=0).stop 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 shutdown_connections 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb7e4147c50 0x7fb7e4149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb7d00683b0 0x7fb7d006a870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7e4148620 0x7fb7e4146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 --2- 172.21.15.40:0/653542374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7e406c8a0 0x7fb7e4072460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 >> 172.21.15.40:0/653542374 conn(0x7fb7e40b8fd0 msgr2=0x7fb7e406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 shutdown_connections 2024-04-02T16:12:04.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.248+0000 7fb7ea99a640 1 -- 172.21.15.40:0/653542374 wait complete. 2024-04-02T16:12:04.266 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T16:12:04.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 Processor -- start 2024-04-02T16:12:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 -- start start 2024-04-02T16:12:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff424147c50 0x7ff4241480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff424148620 0x7ff42414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff424148a90 con 0x7ff42406c8a0 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff4240b2b50 con 0x7ff424148620 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff4288ba640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff4240b0300 con 0x7ff424147c50 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff424148620 0x7ff42414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff424148620 0x7ff42414f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57494/0 (socket says 172.21.15.40:57494) 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 -- 172.21.15.40:0/4054928416 learned_addr learned my addr 172.21.15.40:0/4054928416 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff421d9b640 1 --2- 172.21.15.40:0/4054928416 >> v2:172.21.15.204:3300/0 conn(0x7ff424147c50 0x7ff4241480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 -- 172.21.15.40:0/4054928416 >> v2:172.21.15.204:3300/0 conn(0x7ff424147c50 msgr2=0x7ff4241480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 --2- 172.21.15.40:0/4054928416 >> v2:172.21.15.204:3300/0 conn(0x7ff424147c50 0x7ff4241480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 -- 172.21.15.40:0/4054928416 >> v1:172.21.15.204:6789/0 conn(0x7ff42406c8a0 legacy=0x7ff4240af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.376+0000 7ff422d9d640 1 -- 172.21.15.40:0/4054928416 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4240ae830 con 0x7ff424148620 2024-04-02T16:12:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff422d9d640 1 --2- 172.21.15.40:0/4054928416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff424148620 0x7ff42414f0b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff42406e9c0 tx=0x7ff41c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8afd3969c46ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42159a640 1 -- 172.21.15.40:0/4054928416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a5620 con 0x7ff424148620 2024-04-02T16:12:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42159a640 1 -- 172.21.15.40:0/4054928416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff41c0a57c0 con 0x7ff424148620 2024-04-02T16:12:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42159a640 1 -- 172.21.15.40:0/4054928416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a5b90 con 0x7ff424148620 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/4054928416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff424148620 msgr2=0x7ff42414f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- 172.21.15.40:0/4054928416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff424148620 0x7ff42414f0b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff42406e9c0 tx=0x7ff41c01c960 comp rx=0 tx=0).stop 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/4054928416 shutdown_connections 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- 172.21.15.40:0/4054928416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff424148620 0x7ff42414f0b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- 172.21.15.40:0/4054928416 >> v2:172.21.15.204:3300/0 conn(0x7ff424147c50 0x7ff4241480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/4054928416 >> 172.21.15.40:0/4054928416 conn(0x7ff4240b8fd0 msgr2=0x7ff4240b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/4054928416 shutdown_connections 2024-04-02T16:12:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/4054928416 wait complete. 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 Processor -- start 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- start start 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff424147c50 0x7ff424144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff424148620 0x7ff42409b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b3130 con 0x7ff42406c8a0 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240b0540 con 0x7ff424148620 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff42409b030 con 0x7ff424147c50 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57504/0 (socket says 172.21.15.40:57504) 2024-04-02T16:12:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 -- 172.21.15.40:0/2635079355 learned_addr learned my addr 172.21.15.40:0/2635079355 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff421d9b640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff424147c50 0x7ff424144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff422d9d640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff424148620 0x7ff42409b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 -- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff424147c50 msgr2=0x7ff424144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff424147c50 0x7ff424144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 -- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff424148620 msgr2=0x7ff42409b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff424148620 0x7ff42409b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 -- 172.21.15.40:0/2635079355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42414ede0 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff42259c640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c450 tx=0x7ff41400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414016020 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4140052d0 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414007260 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4241450e0 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.380+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff424145690 con 0x7ff42406c8a0 2024-04-02T16:12:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4240685f0 con 0x7ff42406c8a0 2024-04-02T16:12:04.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7ff414007400 con 0x7ff42406c8a0 2024-04-02T16:12:04.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff412ffd640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 0x7ff40406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff4140862f0 con 0x7ff42406c8a0 2024-04-02T16:12:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff421d9b640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 0x7ff40406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.384+0000 7ff421d9b640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 0x7ff40406a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff424072b80 tx=0x7ff418008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.388+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7ff414020020 con 0x7ff42406c8a0 2024-04-02T16:12:04.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.584+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff42414f110 con 0x7ff42406c8a0 2024-04-02T16:12:04.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff412ffd640 1 -- 172.21.15.40:0/2635079355 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7ff414005480 con 0x7ff42406c8a0 2024-04-02T16:12:04.588 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:04.588 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:12:04.589 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 msgr2=0x7ff40406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 0x7ff40406a840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff424072b80 tx=0x7ff418008040 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 msgr2=0x7ff4240720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c450 tx=0x7ff41400c910 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 shutdown_connections 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff424147c50 0x7ff424144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7ff404068380 0x7ff40406a840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff424148620 0x7ff42409b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 --2- 172.21.15.40:0/2635079355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42406c8a0 0x7ff4240720d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 >> 172.21.15.40:0/2635079355 conn(0x7ff4240b8fd0 msgr2=0x7ff4240b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 shutdown_connections 2024-04-02T16:12:04.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.588+0000 7ff4288ba640 1 -- 172.21.15.40:0/2635079355 wait complete. 2024-04-02T16:12:04.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:04.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.724+0000 7fc4500a4640 1 Processor -- start 2024-04-02T16:12:04.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.724+0000 7fc4500a4640 1 -- start start 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc4480b23d0 0x7fc4480b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc448148e50 0x7fc448149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4480b19e0 con 0x7fc448073230 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4481497c0 con 0x7fc448148e50 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc4480b05c0 con 0x7fc4480b23d0 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc448148e50 0x7fc448149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc448148e50 0x7fc448149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57514/0 (socket says 172.21.15.40:57514) 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 -- 172.21.15.40:0/2066859792 learned_addr learned my addr 172.21.15.40:0/2066859792 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44de40640 1 --2- 172.21.15.40:0/2066859792 >> v2:172.21.15.204:3300/0 conn(0x7fc4480b23d0 0x7fc4480b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 -- 172.21.15.40:0/2066859792 >> v2:172.21.15.204:3300/0 conn(0x7fc4480b23d0 msgr2=0x7fc4480b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 --2- 172.21.15.40:0/2066859792 >> v2:172.21.15.204:3300/0 conn(0x7fc4480b23d0 0x7fc4480b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 -- 172.21.15.40:0/2066859792 >> v1:172.21.15.204:6789/0 conn(0x7fc448073230 legacy=0x7fc44809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 -- 172.21.15.40:0/2066859792 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4480b08a0 con 0x7fc448148e50 2024-04-02T16:12:04.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44e641640 1 --2- 172.21.15.40:0/2066859792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc448148e50 0x7fc448149280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4480b00c0 tx=0x7fc44001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3b5b09f4bfd20b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44ce3e640 1 -- 172.21.15.40:0/2066859792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400a5800 con 0x7fc448148e50 2024-04-02T16:12:04.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44ce3e640 1 -- 172.21.15.40:0/2066859792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4400a59a0 con 0x7fc448148e50 2024-04-02T16:12:04.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc44ce3e640 1 -- 172.21.15.40:0/2066859792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4400a5d70 con 0x7fc448148e50 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 -- 172.21.15.40:0/2066859792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc448148e50 msgr2=0x7fc448149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 --2- 172.21.15.40:0/2066859792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc448148e50 0x7fc448149280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc4480b00c0 tx=0x7fc44001c920 comp rx=0 tx=0).stop 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 -- 172.21.15.40:0/2066859792 shutdown_connections 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.728+0000 7fc4500a4640 1 --2- 172.21.15.40:0/2066859792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc448148e50 0x7fc448149280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 --2- 172.21.15.40:0/2066859792 >> v2:172.21.15.204:3300/0 conn(0x7fc4480b23d0 0x7fc4480b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/2066859792 >> 172.21.15.40:0/2066859792 conn(0x7fc4480b8fd0 msgr2=0x7fc4480b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/2066859792 shutdown_connections 2024-04-02T16:12:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/2066859792 wait complete. 2024-04-02T16:12:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 Processor -- start 2024-04-02T16:12:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- start start 2024-04-02T16:12:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 0x7fc4481e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 0x7fc4481e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc448152810 con 0x7fc4480b23d0 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc448152690 con 0x7fc448073230 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc448152990 con 0x7fc448148e50 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57518/0 (socket says 172.21.15.40:57518) 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 -- 172.21.15.40:0/1047209341 learned_addr learned my addr 172.21.15.40:0/1047209341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44e641640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 0x7fc4481e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44de40640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 0x7fc4481e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 -- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 msgr2=0x7fc4481e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 0x7fc4481e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 -- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 msgr2=0x7fc4481e12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 0x7fc4481e12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 -- 172.21.15.40:0/1047209341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc448148580 con 0x7fc4480b23d0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44e641640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 0x7fc4481e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44de40640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 0x7fc4481e12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc44d63f640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c450 tx=0x7fc44400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444012830 con 0x7fc4480b23d0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc444017070 con 0x7fc4480b23d0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444012c30 con 0x7fc4480b23d0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4480b8c00 con 0x7fc4480b23d0 2024-04-02T16:12:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4481e1820 con 0x7fc4480b23d0 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4481e0fb0 con 0x7fc4480b23d0 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.732+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc44401c460 con 0x7fc4480b23d0 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.736+0000 7fc43e7fc640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 0x7fc42c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.736+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc444086730 con 0x7fc4480b23d0 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.736+0000 7fc44de40640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 0x7fc42c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.736+0000 7fc44de40640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 0x7fc42c06a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc438009840 tx=0x7fc438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:04.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.740+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc4480b8c00 con 0x7fc4480b23d0 2024-04-02T16:12:04.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc44809df90 con 0x7fc4480b23d0 2024-04-02T16:12:04.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc43e7fc640 1 -- 172.21.15.40:0/1047209341 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+3524 (secure 0 0 0) 0x7fc444056cf0 con 0x7fc4480b23d0 2024-04-02T16:12:04.933 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:04.933 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/856008412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":856008412},{"type":"v1","addr":"172.21.15.40:6833","nonce":856008412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:11:31.473429+0000","modified":"2024-04-02T16:11:34.552731+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/905036532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":905036532},{"type":"v1","addr":"172.21.15.204:6835","nonce":905036532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4229,"qdb_cluster":[4229]},"id":1}]} 2024-04-02T16:12:04.934 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-04-02T16:12:04.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 msgr2=0x7fc42c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 0x7fc42c06a870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc438009840 tx=0x7fc438008040 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 msgr2=0x7fc44813ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc44400c450 tx=0x7fc44400c910 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 shutdown_connections 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc448148e50 0x7fc4481e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc42c0683b0 0x7fc42c06a870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4480b23d0 0x7fc44813ccb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 --2- 172.21.15.40:0/1047209341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc448073230 0x7fc4481e12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 >> 172.21.15.40:0/1047209341 conn(0x7fc4480b8fd0 msgr2=0x7fc44806cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 shutdown_connections 2024-04-02T16:12:04.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:04.932+0000 7fc4500a4640 1 -- 172.21.15.40:0/1047209341 wait complete. 2024-04-02T16:12:04.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.068+0000 7f75778ee640 1 Processor -- start 2024-04-02T16:12:05.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.068+0000 7f75778ee640 1 -- start start 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.068+0000 7f75778ee640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f757006d130 0x7f757009dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 0x7f75700b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7570143d10 con 0x7f757009e1f0 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f75700ae7c0 con 0x7f757006d130 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f75700af060 con 0x7f75700b4e90 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7575e8b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 0x7f75700b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f757006d130 0x7f757009dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7575e8b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 0x7f75700b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:55022/0 (socket says 172.21.15.40:55022) 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7575e8b640 1 -- 172.21.15.40:0/3603151207 learned_addr learned my addr 172.21.15.40:0/3603151207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 -- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 msgr2=0x7f75700b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 --2- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 0x7f75700b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 -- 172.21.15.40:0/3603151207 >> v1:172.21.15.204:6789/0 conn(0x7f757009e1f0 legacy=0x7f7570144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 -- 172.21.15.40:0/3603151207 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757006d510 con 0x7f757006d130 2024-04-02T16:12:05.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f757568a640 1 --2- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757006d130 0x7f757009dcb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cfb058fd4a571b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7567fff640 1 -- 172.21.15.40:0/3603151207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b2070 con 0x7f757006d130 2024-04-02T16:12:05.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7567fff640 1 -- 172.21.15.40:0/3603151207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c0a9070 con 0x7f757006d130 2024-04-02T16:12:05.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f7567fff640 1 -- 172.21.15.40:0/3603151207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0ad880 con 0x7f757006d130 2024-04-02T16:12:05.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 -- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757006d130 msgr2=0x7f757009dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.072+0000 7f75778ee640 1 --2- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757006d130 0x7f757009dcb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c3e0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3603151207 shutdown_connections 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 --2- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75700b4e90 0x7f75700b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 --2- 172.21.15.40:0/3603151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f757006d130 0x7f757009dcb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3603151207 >> 172.21.15.40:0/3603151207 conn(0x7f75700b8fd0 msgr2=0x7f75700b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3603151207 shutdown_connections 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3603151207 wait complete. 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 Processor -- start 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- start start 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757009e1f0 0x7f757009af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700b4e90 0x7f75701e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75701505c0 con 0x7f757006d130 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75700ae300 con 0x7f75700b4e90 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7570150740 con 0x7f757009e1f0 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57524/0 (socket says 172.21.15.40:57524) 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 -- 172.21.15.40:0/3469032205 learned_addr learned my addr 172.21.15.40:0/3469032205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f7575e8b640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700b4e90 0x7f75701e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f7574e89640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757009e1f0 0x7f757009af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 -- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757009e1f0 msgr2=0x7f757009af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757009e1f0 0x7f757009af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 -- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700b4e90 msgr2=0x7f75701e05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700b4e90 0x7f75701e05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 -- 172.21.15.40:0/3469032205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701e00f0 con 0x7f757006d130 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f757568a640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f756c0b0040 tx=0x7f756c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b2050 con 0x7f757006d130 2024-04-02T16:12:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c0b3040 con 0x7f757006d130 2024-04-02T16:12:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7570140020 con 0x7f757006d130 2024-04-02T16:12:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c006e20 con 0x7f757006d130 2024-04-02T16:12:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75700b1b50 con 0x7f757006d130 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.076+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75700685f0 con 0x7f757006d130 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.080+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f756c0c0030 con 0x7f757006d130 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.080+0000 7f7565ffb640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.080+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f756c120640 con 0x7f757006d130 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.080+0000 7f7574e89640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.080+0000 7f7574e89640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 0x7f755406a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75700b9420 tx=0x7f7560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.084+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f756c0c7020 con 0x7f757006d130 2024-04-02T16:12:05.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.248+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75700ade20 con 0x7f757006d130 2024-04-02T16:12:05.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f7565ffb640 1 -- 172.21.15.40:0/3469032205 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16557 (secure 0 0 0) 0x7f756c016090 con 0x7f757006d130 2024-04-02T16:12:05.253 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:05.253 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:11:33.147437+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:11:29.810061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T16:11:30.438845+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 msgr2=0x7f755406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 0x7f755406a840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75700b9420 tx=0x7f7560009040 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 msgr2=0x7f757009c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f756c0b0040 tx=0x7f756c01ac20 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 shutdown_connections 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f757009e1f0 0x7f757009af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7554068380 0x7f755406a840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75700b4e90 0x7f75701e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 --2- 172.21.15.40:0/3469032205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f757006d130 0x7f757009c110 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 >> 172.21.15.40:0/3469032205 conn(0x7f75700b8fd0 msgr2=0x7f75701409f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:05.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 shutdown_connections 2024-04-02T16:12:05.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.252+0000 7f75778ee640 1 -- 172.21.15.40:0/3469032205 wait complete. 2024-04-02T16:12:05.273 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T16:12:05.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 Processor -- start 2024-04-02T16:12:05.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 -- start start 2024-04-02T16:12:05.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc638147c50 0x7fc6381480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc638148620 0x7fc63814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6380ae830 con 0x7fc63806c8a0 2024-04-02T16:12:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc6380b2b50 con 0x7fc638147c50 2024-04-02T16:12:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63f40a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380b0300 con 0x7fc638148620 2024-04-02T16:12:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63c9a5640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc638147c50 0x7fc6381480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63d1a6640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc63806c8a0 0x7fc6380af150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:41072/0 (socket says 172.21.15.40:41072) 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63d9a7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc638148620 0x7fc63814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63d1a6640 1 -- 172.21.15.40:0/665931195 learned_addr learned my addr 172.21.15.40:0/665931195 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63c9a5640 1 -- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc638148620 msgr2=0x7fc63814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63c9a5640 1 --2- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc638148620 0x7fc63814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63c9a5640 1 -- 172.21.15.40:0/665931195 >> v1:172.21.15.204:6789/0 conn(0x7fc63806c8a0 legacy=0x7fc6380af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.392+0000 7fc63c9a5640 1 -- 172.21.15.40:0/665931195 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc638148a60 con 0x7fc638147c50 2024-04-02T16:12:05.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63c9a5640 1 --2- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc6381480e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097c0 tx=0x7fc62801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc232365dd483ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc62ffff640 1 -- 172.21.15.40:0/665931195 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280b0070 con 0x7fc638147c50 2024-04-02T16:12:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc62ffff640 1 -- 172.21.15.40:0/665931195 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6280ab600 con 0x7fc638147c50 2024-04-02T16:12:05.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc62ffff640 1 -- 172.21.15.40:0/665931195 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6280ab9d0 con 0x7fc638147c50 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 -- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 msgr2=0x7fc6381480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 --2- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc6381480e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc6280097c0 tx=0x7fc62801c380 comp rx=0 tx=0).stop 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 -- 172.21.15.40:0/665931195 shutdown_connections 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 --2- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc638148620 0x7fc63814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 --2- 172.21.15.40:0/665931195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc6381480e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 -- 172.21.15.40:0/665931195 >> 172.21.15.40:0/665931195 conn(0x7fc6380b8fd0 msgr2=0x7fc6380b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 -- 172.21.15.40:0/665931195 shutdown_connections 2024-04-02T16:12:05.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.396+0000 7fc63f40a640 1 -- 172.21.15.40:0/665931195 wait complete. 2024-04-02T16:12:05.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 Processor -- start 2024-04-02T16:12:05.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- start start 2024-04-02T16:12:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc63806c8a0 0x7fc638143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc63809c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b3130 con 0x7fc638148620 2024-04-02T16:12:05.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b0540 con 0x7fc638147c50 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc63809bf40 con 0x7fc63806c8a0 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39758/0 (socket says 172.21.15.40:39758) 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 -- 172.21.15.40:0/2962822832 learned_addr learned my addr 172.21.15.40:0/2962822832 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63c9a5640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc63809c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 -- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc63806c8a0 msgr2=0x7fc638143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc63806c8a0 0x7fc638143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 -- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 msgr2=0x7fc63809c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc63809c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 -- 172.21.15.40:0/2962822832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6381497e0 con 0x7fc638148620 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63c9a5640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc63809c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63d9a7640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc63000efc0 tx=0x7fc63000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630018020 con 0x7fc638148620 2024-04-02T16:12:05.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc630014480 con 0x7fc638148620 2024-04-02T16:12:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630014870 con 0x7fc638148620 2024-04-02T16:12:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc638149ac0 con 0x7fc638148620 2024-04-02T16:12:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63809d870 con 0x7fc638148620 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6380685f0 con 0x7fc638148620 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.400+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc63001b020 con 0x7fc638148620 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.404+0000 7fc62dffb640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 0x7fc61c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.404+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc630087730 con 0x7fc638148620 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.404+0000 7fc63d1a6640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 0x7fc61c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.404+0000 7fc63d1a6640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 0x7fc61c06a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc634001600 tx=0x7fc634007a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.408+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc630022030 con 0x7fc638148620 2024-04-02T16:12:05.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.608+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc638071bc0 con 0x7fc638148620 2024-04-02T16:12:05.610 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-02T16:12:05.608+0000 7f8eb3a99640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T16:12:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc62dffb640 1 -- 172.21.15.40:0/2962822832 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fc630010040 con 0x7fc638148620 2024-04-02T16:12:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 msgr2=0x7fc61c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 0x7fc61c06a840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc634001600 tx=0x7fc634007a50 comp rx=0 tx=0).stop 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 msgr2=0x7fc63809c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc63000efc0 tx=0x7fc63000c490 comp rx=0 tx=0).stop 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 shutdown_connections 2024-04-02T16:12:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc61c068380 0x7fc61c06a840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc638148620 0x7fc63809c910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc638147c50 0x7fc63809c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 --2- 172.21.15.40:0/2962822832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc63806c8a0 0x7fc638143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.612+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 >> 172.21.15.40:0/2962822832 conn(0x7fc6380b8fd0 msgr2=0x7fc6380b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.616+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 shutdown_connections 2024-04-02T16:12:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.616+0000 7fc63f40a640 1 -- 172.21.15.40:0/2962822832 wait complete. 2024-04-02T16:12:05.625 INFO:tasks.ceph.mds.b.smithi204.stderr: -1267> 2024-04-02T16:11:36.337+0000 7f366df73640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:12:05.633 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T16:12:05.676 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-02T16:12:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.752+0000 7fc443c10640 1 Processor -- start 2024-04-02T16:12:05.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.752+0000 7fc443c10640 1 -- start start 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc443c10640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 0x7fc43c145910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc443c10640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c1462e0 con 0x7fc43c145e50 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc443c10640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c0b5980 con 0x7fc43c02c460 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc443c10640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c0b4570 con 0x7fc43c0b3fc0 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4421ad640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc43c145e50 0x7fc43c14ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44032/0 (socket says 172.21.15.40:44032) 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4421ad640 1 -- 172.21.15.40:0/3273685498 learned_addr learned my addr 172.21.15.40:0/3273685498 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 0x7fc43c145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566733948 0 0) 0x7fc43c1462e0 con 0x7fc43c145e50 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc428003530 con 0x7fc43c145e50 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1866077918 0 0) 0x7fc428003530 con 0x7fc43c145e50 2024-04-02T16:12:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 msgr2=0x7fc43c145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 0x7fc43c145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 >> v1:172.21.15.204:6789/0 conn(0x7fc43c02c460 legacy=0x7fc43c13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c0b4850 con 0x7fc43c145e50 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fc434001680 con 0x7fc43c145e50 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 >> v1:172.21.15.40:6789/0 conn(0x7fc43c145e50 legacy=0x7fc43c14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 0x7fc4280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428003f70 0x7fc428004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428004d50 0x7fc4280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c0b4570 con 0x7fc428003f70 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc42800bab0 con 0x7fc428004d50 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc42800bcb0 con 0x7fc43c0b3fc0 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428003f70 0x7fc428004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4421ad640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428004d50 0x7fc4280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 -- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 msgr2=0x7fc4280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c0b3fc0 0x7fc4280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 -- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428004d50 msgr2=0x7fc4280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428004d50 0x7fc4280051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 -- 172.21.15.40:0/3273685498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc428003530 con 0x7fc428003f70 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428003f70 0x7fc428004810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009ae0 tx=0x7fc42c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317820cac58211b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0b2070 con 0x7fc428003f70 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0a56a0 con 0x7fc428003f70 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.756+0000 7fc4409aa640 1 -- 172.21.15.40:0/3273685498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0096b0 con 0x7fc428003f70 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3273685498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428003f70 msgr2=0x7fc428004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 --2- 172.21.15.40:0/3273685498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc428003f70 0x7fc428004810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009ae0 tx=0x7fc42c01cab0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 -- 172.21.15.40:0/3273685498 reap_dead start 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3273685498 shutdown_connections 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3273685498 >> 172.21.15.40:0/3273685498 conn(0x7fc43c09b370 msgr2=0x7fc43c09b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3273685498 shutdown_connections 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3273685498 wait complete. 2024-04-02T16:12:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 Processor -- start 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- start start 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428003f70 0x7fc43c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 0x7fc43c09dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c0702c0 con 0x7fc43c09ef70 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c13fcc0 con 0x7fc428003f70 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c0b5580 con 0x7fc43c09e6f0 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39768/0 (socket says 172.21.15.40:39768) 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 -- 172.21.15.40:0/3946808188 learned_addr learned my addr 172.21.15.40:0/3946808188 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4419ac640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428003f70 0x7fc43c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 0x7fc43c09dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 -- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 msgr2=0x7fc43c09dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 0x7fc43c09dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 -- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428003f70 msgr2=0x7fc43c140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428003f70 0x7fc43c140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 -- 172.21.15.40:0/3946808188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c1404b0 con 0x7fc43c09ef70 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4411ab640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 0x7fc43c09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4421ad640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc434000c10 tx=0x7fc434010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43401a070 con 0x7fc43c09ef70 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434011040 con 0x7fc43c09ef70 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c0b0340 con 0x7fc43c09ef70 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43400ece0 con 0x7fc43c09ef70 2024-04-02T16:12:05.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c0afb20 con 0x7fc43c09ef70 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.760+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c140230 con 0x7fc43c09ef70 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.764+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc43401d070 con 0x7fc43c09ef70 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.764+0000 7fc4327fc640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 0x7fc42006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.764+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7fc434088f30 con 0x7fc43c09ef70 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.764+0000 7fc4419ac640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 0x7fc42006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:05.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.764+0000 7fc4419ac640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 0x7fc42006a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4380014f0 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:05.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.768+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc43c140230 con 0x7fc43c09ef70 2024-04-02T16:12:05.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:05.960+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc43c0b84d0 con 0x7fc43c09ef70 2024-04-02T16:12:06.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.612+0000 7fc4327fc640 1 -- 172.21.15.40:0/3946808188 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fc434010e60 con 0x7fc43c09ef70 2024-04-02T16:12:06.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 msgr2=0x7fc42006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:06.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 0x7fc42006a840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4380014f0 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-04-02T16:12:06.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 msgr2=0x7fc43c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:06.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc434000c10 tx=0x7fc434010c10 comp rx=0 tx=0).stop 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 shutdown_connections 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc43c09e6f0 0x7fc43c09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc420068380 0x7fc42006a840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc43c09ef70 0x7fc43c0ae5e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 --2- 172.21.15.40:0/3946808188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc428003f70 0x7fc43c140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 >> 172.21.15.40:0/3946808188 conn(0x7fc43c09b370 msgr2=0x7fc43c144490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 shutdown_connections 2024-04-02T16:12:06.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.616+0000 7fc443c10640 1 -- 172.21.15.40:0/3946808188 wait complete. 2024-04-02T16:12:06.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T16:12:06.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 Processor -- start 2024-04-02T16:12:06.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 -- start start 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80bda00 0x7f86c81415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c8141ae0 0x7f86c80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f86c80b5370 0x7f86c814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b7340 con 0x7f86c80bda00 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f86c80b5b50 con 0x7f86c80b5370 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.772+0000 7f86cf7dc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80bdde0 con 0x7f86c8141ae0 2024-04-02T16:12:06.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80bda00 0x7f86c81415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c80bda00 0x7f86c81415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39782/0 (socket says 172.21.15.40:39782) 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 -- 172.21.15.40:0/3699257367 learned_addr learned my addr 172.21.15.40:0/3699257367 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86ccd77640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c8141ae0 0x7f86c80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cdd79640 1 --2- 172.21.15.40:0/3699257367 >> v2:172.21.15.204:3300/0 conn(0x7f86c80b5370 0x7f86c814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 -- 172.21.15.40:0/3699257367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c8141ae0 msgr2=0x7f86c80b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c8141ae0 0x7f86c80b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 -- 172.21.15.40:0/3699257367 >> v2:172.21.15.204:3300/0 conn(0x7f86c80b5370 msgr2=0x7f86c814ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 --2- 172.21.15.40:0/3699257367 >> v2:172.21.15.204:3300/0 conn(0x7f86c80b5370 0x7f86c814ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 -- 172.21.15.40:0/3699257367 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c80b5df0 con 0x7f86c80bda00 2024-04-02T16:12:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86cd578640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80bda00 0x7f86c81415a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b4540 tx=0x7f86c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ea91f6ac82176a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86bffff640 1 -- 172.21.15.40:0/3699257367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40ab600 con 0x7f86c80bda00 2024-04-02T16:12:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86bffff640 1 -- 172.21.15.40:0/3699257367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c40b0070 con 0x7f86c80bda00 2024-04-02T16:12:06.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.776+0000 7f86bffff640 1 -- 172.21.15.40:0/3699257367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c40ab9c0 con 0x7f86c80bda00 2024-04-02T16:12:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/3699257367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80bda00 msgr2=0x7f86c81415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80bda00 0x7f86c81415a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f86c80b4540 tx=0x7f86c401c460 comp rx=0 tx=0).stop 2024-04-02T16:12:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/3699257367 shutdown_connections 2024-04-02T16:12:06.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/3699257367 >> v2:172.21.15.204:3300/0 conn(0x7f86c80b5370 0x7f86c814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c8141ae0 0x7f86c80b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/3699257367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80bda00 0x7f86c81415a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/3699257367 >> 172.21.15.40:0/3699257367 conn(0x7f86c8073340 msgr2=0x7f86c80bd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:06.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/3699257367 shutdown_connections 2024-04-02T16:12:06.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/3699257367 wait complete. 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 Processor -- start 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- start start 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c80b5370 0x7f86c813e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c80bda00 0x7f86c81e0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86c80b85b0 con 0x7f86c8141ae0 2024-04-02T16:12:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86c80b8430 con 0x7f86c80bda00 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f86c80b8730 con 0x7f86c80b5370 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39794/0 (socket says 172.21.15.40:39794) 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 -- 172.21.15.40:0/2497898512 learned_addr learned my addr 172.21.15.40:0/2497898512 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86ccd77640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c80bda00 0x7f86c81e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cd578640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c80b5370 0x7f86c813e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 -- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c80b5370 msgr2=0x7f86c813e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c80b5370 0x7f86c813e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 -- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c80bda00 msgr2=0x7f86c81e0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c80bda00 0x7f86c81e0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 -- 172.21.15.40:0/2497898512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c813e290 con 0x7f86c8141ae0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cdd79640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86c0007d20 tx=0x7f86c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0014420 con 0x7f86c8141ae0 2024-04-02T16:12:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c0019070 con 0x7f86c8141ae0 2024-04-02T16:12:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0014820 con 0x7f86c8141ae0 2024-04-02T16:12:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c813d4c0 con 0x7f86c8141ae0 2024-04-02T16:12:06.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.780+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c81481f0 con 0x7f86c8141ae0 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c80685f0 con 0x7f86c8141ae0 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f86c00149c0 con 0x7f86c8141ae0 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86bdffb640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 0x7f86ac06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f86c0087030 con 0x7f86c8141ae0 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86cd578640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 0x7f86ac06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.784+0000 7f86cd578640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 0x7f86ac06a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86c40af000 tx=0x7f86c40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:06.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.788+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f86c0056fd0 con 0x7f86c8141ae0 2024-04-02T16:12:06.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:06.952+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86c8148610 con 0x7f86c8141ae0 2024-04-02T16:12:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.620+0000 7f86bdffb640 1 -- 172.21.15.40:0/2497898512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f86c005b2d0 con 0x7f86c8141ae0 2024-04-02T16:12:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-02T16:12:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 msgr2=0x7f86ac06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 0x7f86ac06a840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86c40af000 tx=0x7f86c40a7000 comp rx=0 tx=0).stop 2024-04-02T16:12:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 msgr2=0x7f86c81e0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86c0007d20 tx=0x7f86c000e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 shutdown_connections 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f86ac068380 0x7f86ac06a840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8141ae0 0x7f86c81e0dc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c80bda00 0x7f86c81e0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 --2- 172.21.15.40:0/2497898512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c80b5370 0x7f86c813e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.624+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 >> 172.21.15.40:0/2497898512 conn(0x7f86c8073340 msgr2=0x7f86c80b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.628+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 shutdown_connections 2024-04-02T16:12:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.628+0000 7f86cf7dc640 1 -- 172.21.15.40:0/2497898512 wait complete. 2024-04-02T16:12:07.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T16:12:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 Processor -- start 2024-04-02T16:12:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 -- start start 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa01814dde0 0x7fa0181501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa018148090 con 0x7fa01813e170 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01806da50 con 0x7fa018147cb0 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01f669640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa01806dd30 con 0x7fa01814dde0 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01cc04640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa01813e170 0x7fa01813e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44036/0 (socket says 172.21.15.40:44036) 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01cc04640 1 -- 172.21.15.40:0/3353026824 learned_addr learned my addr 172.21.15.40:0/3353026824 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa01dc06640 1 --2- 172.21.15.40:0/3353026824 >> v2:172.21.15.204:3300/0 conn(0x7fa01814dde0 0x7fa0181501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.784+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075712134 0 0) 0x7fa018148090 con 0x7fa01813e170 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa004003530 con 0x7fa01813e170 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 315625435 0 0) 0x7fa004003530 con 0x7fa01813e170 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 >> v2:172.21.15.204:3300/0 conn(0x7fa01814dde0 msgr2=0x7fa0181501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 --2- 172.21.15.40:0/3353026824 >> v2:172.21.15.204:3300/0 conn(0x7fa01814dde0 0x7fa0181501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 >> v1:172.21.15.204:6789/0 conn(0x7fa018147cb0 legacy=0x7fa01806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01806e010 con 0x7fa01813e170 2024-04-02T16:12:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fa014001680 con 0x7fa01813e170 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 >> v1:172.21.15.40:6789/0 conn(0x7fa01813e170 legacy=0x7fa01813e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa004003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa004004810 0x7fa004004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0040051d0 0x7fa00400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa01806dd30 con 0x7fa0040039c0 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa00400c170 con 0x7fa0040051d0 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa00400c370 con 0x7fa004004810 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa004003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01dc06640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0040051d0 0x7fa00400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 -- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa004004810 msgr2=0x7fa004004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa004004810 0x7fa004004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 -- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0040051d0 msgr2=0x7fa00400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:07.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0040051d0 0x7fa00400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01dc06640 1 -- 172.21.15.40:0/3353026824 reap_dead start 2024-04-02T16:12:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 -- 172.21.15.40:0/3353026824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa004003530 con 0x7fa0040039c0 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01d405640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa004003de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001360 tx=0x7fa00c01e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e417f12c30ef6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0b2070 con 0x7fa0040039c0 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa00c0052f0 con 0x7fa0040039c0 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa00bfff640 1 -- 172.21.15.40:0/3353026824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0013a0 con 0x7fa0040039c0 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 -- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 msgr2=0x7fa004003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa004003de0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001360 tx=0x7fa00c01e610 comp rx=0 tx=0).stop 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 -- 172.21.15.40:0/3353026824 shutdown_connections 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 --2- 172.21.15.40:0/3353026824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa004003de0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 -- 172.21.15.40:0/3353026824 >> 172.21.15.40:0/3353026824 conn(0x7fa0180b7d80 msgr2=0x7fa0180b8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 -- 172.21.15.40:0/3353026824 shutdown_connections 2024-04-02T16:12:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.788+0000 7fa01f669640 1 -- 172.21.15.40:0/3353026824 wait complete. 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 Processor -- start 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- start start 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 0x7fa018144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa018144850 0x7fa0181e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181488f0 con 0x7fa0040039c0 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa018148770 con 0x7fa018142cb0 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa01806e530 con 0x7fa018144850 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39810/0 (socket says 172.21.15.40:39810) 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 -- 172.21.15.40:0/4135721223 learned_addr learned my addr 172.21.15.40:0/4135721223 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01cc04640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 0x7fa018144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01dc06640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa018144850 0x7fa0181e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 -- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa018144850 msgr2=0x7fa0181e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa018144850 0x7fa0181e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 -- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 msgr2=0x7fa018144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 0x7fa018144310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 -- 172.21.15.40:0/4135721223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180732b0 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01cc04640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 0x7fa018144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01d405640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa00c024b40 tx=0x7fa004005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0bb070 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa00c0054a0 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa018144cb0 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0b2040 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa018143470 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.792+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0180685f0 con 0x7fa0040039c0 2024-04-02T16:12:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.796+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa00c0be070 con 0x7fa0040039c0 2024-04-02T16:12:07.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.796+0000 7fa009ffb640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 0x7f9ffc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:07.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.796+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7fa00c11fc20 con 0x7fa0040039c0 2024-04-02T16:12:07.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.796+0000 7fa01cc04640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 0x7f9ffc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:07.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.796+0000 7fa01cc04640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 0x7f9ffc06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa014007a80 tx=0x7fa014009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:07.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.800+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa00c018320 con 0x7fa0040039c0 2024-04-02T16:12:07.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:07.960+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa0180b0570 con 0x7fa0040039c0 2024-04-02T16:12:08.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.632+0000 7fa009ffb640 1 -- 172.21.15.40:0/4135721223 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7fa00c0efce0 con 0x7fa0040039c0 2024-04-02T16:12:08.634 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 msgr2=0x7f9ffc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 0x7f9ffc06a840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa014007a80 tx=0x7fa014009040 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 msgr2=0x7fa0180b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa00c024b40 tx=0x7fa004005840 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 shutdown_connections 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa018144850 0x7fa0181e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9ffc068380 0x7f9ffc06a840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa018142cb0 0x7fa018144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 --2- 172.21.15.40:0/4135721223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0040039c0 0x7fa0180b6330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 >> 172.21.15.40:0/4135721223 conn(0x7fa0180b7d80 msgr2=0x7fa01806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:08.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 shutdown_connections 2024-04-02T16:12:08.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.636+0000 7fa01f669640 1 -- 172.21.15.40:0/4135721223 wait complete. 2024-04-02T16:12:08.664 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-02T16:12:08.664 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-02T16:12:08.665 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-02T16:12:08.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:12:08.665 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-02T16:12:08.665 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-02T16:12:08.665 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-02T16:12:08.666 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:12:08.676 INFO:tasks.ceph.mds.a:Stopped 2024-04-02T16:12:08.676 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-02T16:12:08.681 INFO:tasks.ceph.mds.b:Stopped 2024-04-02T16:12:08.681 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-02T16:12:08.721 INFO:tasks.ceph.mds.a:Started 2024-04-02T16:12:08.721 DEBUG:teuthology.parallel:result is None 2024-04-02T16:12:08.725 INFO:tasks.ceph.mds.b:Started 2024-04-02T16:12:08.725 DEBUG:teuthology.parallel:result is None 2024-04-02T16:12:08.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-02T16:12:08.796 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-02T16:12:08.796 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-04-02T16:12:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 Processor -- start 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 -- start start 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4ff00af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff0071550 con 0x7f4ff0072910 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ff0147a70 con 0x7f4ff0148550 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4ff600e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff0147d50 con 0x7f4ff0141b70 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4fef7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4ff0072910 0x7f4ff0070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44042/0 (socket says 172.21.15.40:44042) 2024-04-02T16:12:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4fef7fe640 1 -- 172.21.15.40:0/3893129916 learned_addr learned my addr 172.21.15.40:0/3893129916 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:08.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4ff00af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515069928 0 0) 0x7f4ff0071550 con 0x7f4ff0072910 2024-04-02T16:12:08.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fdc005740 con 0x7f4ff0072910 2024-04-02T16:12:08.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554812433 0 0) 0x7f4ff0147a70 con 0x7f4ff0148550 2024-04-02T16:12:08.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ff0071550 con 0x7f4ff0148550 2024-04-02T16:12:08.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1484243013 0 0) 0x7f4fdc005740 con 0x7f4ff0072910 2024-04-02T16:12:08.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 msgr2=0x7f4ff00af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:08.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4ff00af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 >> v1:172.21.15.204:6789/0 conn(0x7f4ff0148550 legacy=0x7f4ff0141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff0148030 con 0x7f4ff0072910 2024-04-02T16:12:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f4fe00016a0 con 0x7f4ff0072910 2024-04-02T16:12:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 >> v1:172.21.15.40:6789/0 conn(0x7f4ff0072910 legacy=0x7f4ff0070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4fdc005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fdc0062f0 0x7f4fdc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fdc006f60 0x7f4fdc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff0147d50 con 0x7f4fdc006f60 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc00ddf0 con 0x7f4fdc0062f0 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc00dff0 con 0x7f4ff0141b70 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fdc006f60 0x7f4fdc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4fef7fe640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4fdc005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4fe7fff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fdc0062f0 0x7f4fdc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 -- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 msgr2=0x7f4fdc005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff0141b70 0x7f4fdc005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 -- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fdc0062f0 msgr2=0x7f4fdc006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fdc0062f0 0x7f4fdc006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.848+0000 7f4feffff640 1 -- 172.21.15.40:0/3893129916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc005740 con 0x7f4fdc006f60 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feffff640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fdc006f60 0x7f4fdc0073e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009fa0 tx=0x7f4fe801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba06c5753d030079 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80b2070 con 0x7f4fdc006f60 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe80a5b00 con 0x7f4fdc006f60 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feeffd640 1 -- 172.21.15.40:0/3893129916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8009bf0 con 0x7f4fdc006f60 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- 172.21.15.40:0/3893129916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fdc006f60 msgr2=0x7f4fdc0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 --2- 172.21.15.40:0/3893129916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fdc006f60 0x7f4fdc0073e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009fa0 tx=0x7f4fe801cc40 comp rx=0 tx=0).stop 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feffff640 1 -- 172.21.15.40:0/3893129916 reap_dead start 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- 172.21.15.40:0/3893129916 shutdown_connections 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- 172.21.15.40:0/3893129916 >> 172.21.15.40:0/3893129916 conn(0x7f4ff00b7f50 msgr2=0x7f4ff00b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- 172.21.15.40:0/3893129916 shutdown_connections 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- 172.21.15.40:0/3893129916 wait complete. 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 Processor -- start 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- start start 2024-04-02T16:12:08.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff01306e0 0x7f4ff0131240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff0132080 0x7f4ff012d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff006bf10 con 0x7f4ff0131c00 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff006bd90 con 0x7f4ff0132080 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4ff600e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff006c090 con 0x7f4ff01306e0 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39860/0 (socket says 172.21.15.40:39860) 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 -- 172.21.15.40:0/2560278996 learned_addr learned my addr 172.21.15.40:0/2560278996 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4feffff640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff0132080 0x7f4ff012d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fef7fe640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff01306e0 0x7f4ff0131240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 -- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff01306e0 msgr2=0x7f4ff0131240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff01306e0 0x7f4ff0131240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 -- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff0132080 msgr2=0x7f4ff012d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff0132080 0x7f4ff012d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 -- 172.21.15.40:0/2560278996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff0130ac0 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fe7fff640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e7b0 tx=0x7f4fd800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.852+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd80050d0 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd8018070 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff006df50 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800c6c0 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff01358e0 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff0068d00 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4fd800c860 con 0x7f4ff0131c00 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fecff9640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 0x7f4fd006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:08.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f4fd8086390 con 0x7f4ff0131c00 2024-04-02T16:12:08.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fef7fe640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 0x7f4fd006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:08.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.856+0000 7f4fef7fe640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 0x7f4fd006a870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0001230 tx=0x7f4fe0000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:08.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:08.860+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f4fd8056da0 con 0x7f4ff0131c00 2024-04-02T16:12:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.048+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4ff0073050 con 0x7f4ff0131c00 2024-04-02T16:12:09.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.652+0000 7f4fecff9640 1 -- 172.21.15.40:0/2560278996 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f4fd80566a0 con 0x7f4ff0131c00 2024-04-02T16:12:09.653 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 msgr2=0x7f4fd006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 0x7f4fd006a870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0001230 tx=0x7f4fe0000ee0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 msgr2=0x7f4ff012cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e7b0 tx=0x7f4fd800ec70 comp rx=0 tx=0).stop 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 shutdown_connections 2024-04-02T16:12:09.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4fd00683b0 0x7f4fd006a870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ff0132080 0x7f4ff012d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0131c00 0x7f4ff012cef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 --2- 172.21.15.40:0/2560278996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ff01306e0 0x7f4ff0131240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 >> 172.21.15.40:0/2560278996 conn(0x7f4ff00b7f50 msgr2=0x7f4ff013bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 shutdown_connections 2024-04-02T16:12:09.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.656+0000 7f4ff600e640 1 -- 172.21.15.40:0/2560278996 wait complete. 2024-04-02T16:12:09.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-02T16:12:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 Processor -- start 2024-04-02T16:12:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 -- start start 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2d981487f0 0x7f2d981500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d980ae830 con 0x7f2d98147c50 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9806cc80 con 0x7f2d9806c8a0 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9f53f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2d980b2b50 con 0x7f2d981487f0 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9cada640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2d98147c50 0x7f2d981480e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44048/0 (socket says 172.21.15.40:44048) 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9cada640 1 -- 172.21.15.40:0/2667648913 learned_addr learned my addr 172.21.15.40:0/2667648913 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2667648913 >> v2:172.21.15.204:3300/0 conn(0x7f2d981487f0 0x7f2d981500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 81455295 0 0) 0x7f2d980ae830 con 0x7f2d98147c50 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d84003530 con 0x7f2d98147c50 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2130970553 0 0) 0x7f2d84003530 con 0x7f2d98147c50 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 >> v2:172.21.15.204:3300/0 conn(0x7f2d981487f0 msgr2=0x7f2d981500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 --2- 172.21.15.40:0/2667648913 >> v2:172.21.15.204:3300/0 conn(0x7f2d981487f0 0x7f2d981500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 >> v1:172.21.15.204:6789/0 conn(0x7f2d9806c8a0 legacy=0x7f2d980af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d980adfa0 con 0x7f2d98147c50 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f2d88001680 con 0x7f2d98147c50 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 >> v1:172.21.15.40:6789/0 conn(0x7f2d98147c50 legacy=0x7f2d981480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840039c0 0x7f2d84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d84004810 0x7f2d84004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 0x7f2d8400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980b2b50 con 0x7f2d840051d0 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8400c170 con 0x7f2d84004810 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8400c370 con 0x7f2d840039c0 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 0x7f2d8400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9cada640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d84004810 0x7f2d84004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9d2db640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840039c0 0x7f2d84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 -- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840039c0 msgr2=0x7f2d84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840039c0 0x7f2d84003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 -- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d84004810 msgr2=0x7f2d84004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d84004810 0x7f2d84004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.804+0000 7f2d9dadc640 1 -- 172.21.15.40:0/2667648913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84003530 con 0x7f2d840051d0 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9dadc640 1 -- 172.21.15.40:0/2667648913 reap_dead start 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 0x7f2d8400b9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009fc0 tx=0x7f2d9001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7e88059c30e037 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d90009c10 con 0x7f2d840051d0 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d900a5bc0 con 0x7f2d840051d0 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d8ffff640 1 -- 172.21.15.40:0/2667648913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d900039a0 con 0x7f2d840051d0 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 msgr2=0x7f2d8400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 0x7f2d8400b9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009fc0 tx=0x7f2d9001ccc0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2667648913 shutdown_connections 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2667648913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d840051d0 0x7f2d8400b9d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2667648913 >> 172.21.15.40:0/2667648913 conn(0x7f2d980b8fd0 msgr2=0x7f2d980b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2667648913 shutdown_connections 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2667648913 wait complete. 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 Processor -- start 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- start start 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840051d0 0x7f2d98154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d981e05b0 0x7f2d981e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980b3130 con 0x7f2d9813c7c0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980b0300 con 0x7f2d981e05b0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9f53f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2d98070bb0 con 0x7f2d840051d0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9cada640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9cada640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39876/0 (socket says 172.21.15.40:39876) 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.808+0000 7f2d9cada640 1 -- 172.21.15.40:0/2359948429 learned_addr learned my addr 172.21.15.40:0/2359948429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9d2db640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840051d0 0x7f2d98154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9dadc640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d981e05b0 0x7f2d981e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 -- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840051d0 msgr2=0x7f2d98154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840051d0 0x7f2d98154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 -- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d981e05b0 msgr2=0x7f2d981e0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d981e05b0 0x7f2d981e0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 -- 172.21.15.40:0/2359948429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9813d180 con 0x7f2d9813c7c0 2024-04-02T16:12:09.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9cada640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011840 tx=0x7f2d88011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88010040 con 0x7f2d9813c7c0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d8801a070 con 0x7f2d9813c7c0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d98071100 con 0x7f2d9813c7c0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800fd40 con 0x7f2d9813c7c0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9806fa90 con 0x7f2d9813c7c0 2024-04-02T16:12:09.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d980685f0 con 0x7f2d9813c7c0 2024-04-02T16:12:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2d8801f460 con 0x7f2d9813c7c0 2024-04-02T16:12:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 0x7f2d7c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f2d880897a0 con 0x7f2d9813c7c0 2024-04-02T16:12:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9d2db640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 0x7f2d7c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:09.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.812+0000 7f2d9d2db640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 0x7f2d7c06a870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c10 tx=0x7f2d94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:09.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:09.816+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2d9813d180 con 0x7f2d9813c7c0 2024-04-02T16:12:10.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2d98070e20 con 0x7f2d9813c7c0 2024-04-02T16:12:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d8dffb640 1 -- 172.21.15.40:0/2359948429 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+5207 (secure 0 0 0) 0x7f2d8805a0b0 con 0x7f2d9813c7c0 2024-04-02T16:12:10.041 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:10.041 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:10.041 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2024-04-02T16:12:10.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxmFZ4ZLRAAg9UAvB41KyNBsCRcvn5cZQ==", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2024-04-02T16:12:10.042 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxm5UwBMxAA3LvKPHoC1VUToQkOOiE/TA==", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD8LQxmU7rgNhAAFS4pWl9VA3B04Dl7RnTg4A==", 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAALgxmjsN6ARAA0011kcPPwUGqnylR78uhxA==", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQADLgxmkgLVCRAAhUqJntUebAs0li6gWAG4eg==", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAGLgxm4B8GERAAOXhjH8GXhH7jNNZl9+aRRw==", 2024-04-02T16:12:10.045 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAMLgxmKeGnDRAAQXoRx/Cm6ucbtKRfz8rA6A==", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAPLgxmkcgFExAAS94XrJ6sYuw2Jct16snpSQ==", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQASLgxmSde+GhAAvCSikfKuv2IlJtuC0UkOww==", 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.047 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAVLgxm5+2KIRAAq3Mzsnq5IHzVM2PJd4bkjg==", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.048 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxm3/mnOBAA99WD/i8maqhym9P/RBkKdw==", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxmPGEiFhAAZM6px6MIxwWwSXD5OQnlyQ==", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-02T16:12:10.049 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmD8UBChAAuMaDsg0wcDIA32DzRRNCQg==", 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2024-04-02T16:12:10.050 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmA+ABChAAJBr33Vis1qvhtg7mSjMcyw==", 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmtvgBChAAopjYWD4pSKZohuSIAAQAPw==", 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2024-04-02T16:12:10.051 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmDg8CChAAXRgxJwg5o1GpZHPuWBhpxA==", 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmwCUCChAAoyb4bRXLedG3Ynlc9DnqsQ==", 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2024-04-02T16:12:10.052 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQAbLgxmOjwCChAAE1Xz40e9mdjXIy6J/leB/A==", 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxmMyrbKRAAsjswzcBmDGKDF007lGbGKw==", 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:10.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQD5LQxmC8MbJxAACP7SD6VdBN3rfrbHid7szw==", 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-02T16:12:10.054 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:10.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 msgr2=0x7f2d7c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 0x7f2d7c06a870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c10 tx=0x7f2d94009040 comp rx=0 tx=0).stop 2024-04-02T16:12:10.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 msgr2=0x7f2d9813cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d88011840 tx=0x7f2d88011d00 comp rx=0 tx=0).stop 2024-04-02T16:12:10.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 shutdown_connections 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2d7c0683b0 0x7f2d7c06a870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d981e05b0 0x7f2d981e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9813c7c0 0x7f2d9813cc40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 --2- 172.21.15.40:0/2359948429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d840051d0 0x7f2d98154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 >> 172.21.15.40:0/2359948429 conn(0x7f2d980b8fd0 msgr2=0x7f2d9806d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 shutdown_connections 2024-04-02T16:12:10.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.040+0000 7f2d9f53f640 1 -- 172.21.15.40:0/2359948429 wait complete. 2024-04-02T16:12:10.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-02T16:12:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 Processor -- start 2024-04-02T16:12:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 -- start start 2024-04-02T16:12:10.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f2809dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f2809e1f0 0x7f7f28144520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7f28144a60 0x7f7f280b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7f280ae7c0 con 0x7f7f2809e1f0 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7f280af060 con 0x7f7f28144a60 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f2e04b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7f28143d10 con 0x7f7f2806d130 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f2809e1f0 0x7f7f28144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f2809e1f0 0x7f7f28144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39890/0 (socket says 172.21.15.40:39890) 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 -- 172.21.15.40:0/2363118856 learned_addr learned my addr 172.21.15.40:0/2363118856 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f27fff640 1 --2- 172.21.15.40:0/2363118856 >> v2:172.21.15.204:3300/0 conn(0x7f7f28144a60 0x7f7f280b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f277fe640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f2809dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 -- 172.21.15.40:0/2363118856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 msgr2=0x7f7f2809dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f2809dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 -- 172.21.15.40:0/2363118856 >> v2:172.21.15.204:3300/0 conn(0x7f7f28144a60 msgr2=0x7f7f280b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/2363118856 >> v2:172.21.15.204:3300/0 conn(0x7f7f28144a60 0x7f7f280b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.184+0000 7f7f26ffd640 1 -- 172.21.15.40:0/2363118856 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2806d510 con 0x7f7f2809e1f0 2024-04-02T16:12:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f28144520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f280afe30 tx=0x7f7f1001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01112ea01a0ed9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f267fc640 1 -- 172.21.15.40:0/2363118856 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100b1070 con 0x7f7f2809e1f0 2024-04-02T16:12:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f267fc640 1 -- 172.21.15.40:0/2363118856 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f100ac620 con 0x7f7f2809e1f0 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f267fc640 1 -- 172.21.15.40:0/2363118856 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100ac9f0 con 0x7f7f2809e1f0 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- 172.21.15.40:0/2363118856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 msgr2=0x7f7f28144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f28144520 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f280afe30 tx=0x7f7f1001c460 comp rx=0 tx=0).stop 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- 172.21.15.40:0/2363118856 shutdown_connections 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/2363118856 >> v2:172.21.15.204:3300/0 conn(0x7f7f28144a60 0x7f7f280b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f28144520 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/2363118856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f2809dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- 172.21.15.40:0/2363118856 >> 172.21.15.40:0/2363118856 conn(0x7f7f280b8fd0 msgr2=0x7f7f280b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- 172.21.15.40:0/2363118856 shutdown_connections 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- 172.21.15.40:0/2363118856 wait complete. 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 Processor -- start 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- start start 2024-04-02T16:12:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f28070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f28144a60 0x7f7f28140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28147400 con 0x7f7f2809e1f0 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28147280 con 0x7f7f28144a60 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f2e04b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7f28147580 con 0x7f7f2806d130 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39898/0 (socket says 172.21.15.40:39898) 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 -- 172.21.15.40:0/3340724933 learned_addr learned my addr 172.21.15.40:0/3340724933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 -- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 msgr2=0x7f7f28070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f28070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f26ffd640 1 -- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f28144a60 msgr2=0x7f7f28140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.188+0000 7f7f27fff640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f28144a60 0x7f7f28140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f28144a60 0x7f7f28140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f26ffd640 1 -- 172.21.15.40:0/3340724933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f28071430 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f26ffd640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f1001ab10 tx=0x7f7f10019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100b1040 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f100ba070 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281de090 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100b5660 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281e15a0 con 0x7f7f2809e1f0 2024-04-02T16:12:10.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f280685f0 con 0x7f7f2809e1f0 2024-04-02T16:12:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7f100b5800 con 0x7f7f2809e1f0 2024-04-02T16:12:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 0x7f7f0806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f7f10120340 con 0x7f7f2809e1f0 2024-04-02T16:12:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f277fe640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 0x7f7f0806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.192+0000 7f7f277fe640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 0x7f7f0806a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f200070b0 tx=0x7f7f20007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.196+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7f10016110 con 0x7f7f2809e1f0 2024-04-02T16:12:10.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.416+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f7f2806deb0 con 0x7f7f2809e1f0 2024-04-02T16:12:10.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f1ffff640 1 -- 172.21.15.40:0/3340724933 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f7f100f0e00 con 0x7f7f2809e1f0 2024-04-02T16:12:10.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 msgr2=0x7f7f0806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 0x7f7f0806a840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f200070b0 tx=0x7f7f20007040 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 msgr2=0x7f7f281e1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f1001ab10 tx=0x7f7f10019200 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 shutdown_connections 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7f08068380 0x7f7f0806a840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f28144a60 0x7f7f28140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f2809e1f0 0x7f7f281e1da0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 --2- 172.21.15.40:0/3340724933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f2806d130 0x7f7f28070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 >> 172.21.15.40:0/3340724933 conn(0x7f7f280b8fd0 msgr2=0x7f7f281420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 shutdown_connections 2024-04-02T16:12:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.420+0000 7f7f2e04b640 1 -- 172.21.15.40:0/3340724933 wait complete. 2024-04-02T16:12:10.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-02T16:12:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 Processor -- start 2024-04-02T16:12:10.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 -- start start 2024-04-02T16:12:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c02c460 0x7f4e9c0b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4e9c13eab0 0x7f4e9c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e9c0b51f0 con 0x7f4e9c0b8b10 2024-04-02T16:12:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c0b3eb0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.548+0000 7f4ea3316640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c0b4340 con 0x7f4e9c02c460 2024-04-02T16:12:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4e9c13eab0 0x7f4e9c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea10b2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c02c460 0x7f4e9c0b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4e9c13eab0 0x7f4e9c14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:43282/0 (socket says 172.21.15.40:43282) 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 -- 172.21.15.40:0/3091508430 learned_addr learned my addr 172.21.15.40:0/3091508430 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 -- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c02c460 msgr2=0x7f4e9c0b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 --2- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c02c460 0x7f4e9c0b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 -- 172.21.15.40:0/3091508430 >> v1:172.21.15.204:6789/0 conn(0x7f4e9c0b8b10 legacy=0x7f4e9c13e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 -- 172.21.15.40:0/3091508430 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c02c840 con 0x7f4e9c13eab0 2024-04-02T16:12:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea18b3640 1 --2- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c14ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e50 tx=0x7f4e9401ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a87f231743b7a3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4e93fff640 1 -- 172.21.15.40:0/3091508430 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a7780 con 0x7f4e9c13eab0 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4e93fff640 1 -- 172.21.15.40:0/3091508430 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e940a7920 con 0x7f4e9c13eab0 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4e93fff640 1 -- 172.21.15.40:0/3091508430 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a7cf0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea3316640 1 -- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c13eab0 msgr2=0x7f4e9c14ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.552+0000 7f4ea3316640 1 --2- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c14ee00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009e50 tx=0x7f4e9401ebb0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/3091508430 shutdown_connections 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 --2- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c14ee00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 --2- 172.21.15.40:0/3091508430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c02c460 0x7f4e9c0b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/3091508430 >> 172.21.15.40:0/3091508430 conn(0x7f4e9c09b260 msgr2=0x7f4e9c09b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/3091508430 shutdown_connections 2024-04-02T16:12:10.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/3091508430 wait complete. 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 Processor -- start 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- start start 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c02c460 0x7f4e9c09e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 0x7f4e9c09e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c06d840 con 0x7f4e9c13eab0 2024-04-02T16:12:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c06d6c0 con 0x7f4e9c02c460 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c06d9c0 con 0x7f4e9c0b8b10 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39914/0 (socket says 172.21.15.40:39914) 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 -- 172.21.15.40:0/1734309272 learned_addr learned my addr 172.21.15.40:0/1734309272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea08b1640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 0x7f4e9c09e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea10b2640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c02c460 0x7f4e9c09e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 -- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 msgr2=0x7f4e9c09e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 0x7f4e9c09e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 -- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c02c460 msgr2=0x7f4e9c09e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c02c460 0x7f4e9c09e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 -- 172.21.15.40:0/1734309272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c09f060 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea08b1640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 0x7f4e9c09e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea18b3640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4e9401c7e0 tx=0x7f4e9401b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940be070 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e940073f0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c1e4230 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940077c0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c1e3ed0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c0b4950 con 0x7f4e9c13eab0 2024-04-02T16:12:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.556+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f4e940c1070 con 0x7f4e9c13eab0 2024-04-02T16:12:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.560+0000 7f4e91ffb640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 0x7f4e8006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.560+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f4e9411ffe0 con 0x7f4e9c13eab0 2024-04-02T16:12:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.560+0000 7f4ea10b2640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 0x7f4e8006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.560+0000 7f4ea10b2640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 0x7f4e8006a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009ab0 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.564+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f4e94018830 con 0x7f4e9c13eab0 2024-04-02T16:12:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.784+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4e9c068d00 con 0x7f4e9c13eab0 2024-04-02T16:12:10.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4e91ffb640 1 -- 172.21.15.40:0/1734309272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f4e940f0920 con 0x7f4e9c13eab0 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 msgr2=0x7f4e8006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 0x7f4e8006a840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009ab0 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 msgr2=0x7f4e9c1e0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4e9401c7e0 tx=0x7f4e9401b950 comp rx=0 tx=0).stop 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 shutdown_connections 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e9c0b8b10 0x7f4e9c09e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4e80068380 0x7f4e8006a840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e9c13eab0 0x7f4e9c1e0730 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 --2- 172.21.15.40:0/1734309272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e9c02c460 0x7f4e9c09e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 >> 172.21.15.40:0/1734309272 conn(0x7f4e9c09b260 msgr2=0x7f4e9c0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 shutdown_connections 2024-04-02T16:12:10.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.788+0000 7f4ea3316640 1 -- 172.21.15.40:0/1734309272 wait complete. 2024-04-02T16:12:10.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-02T16:12:10.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 Processor -- start 2024-04-02T16:12:10.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 -- start start 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f83f806d130 0x7f83f809dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83f80b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f8143d10 con 0x7f83f809e1f0 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f83f80ae7c0 con 0x7f83f806d130 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83ffdc9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80af060 con 0x7f83f80b4e90 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fd364640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f83f809e1f0 0x7f83f8144520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44054/0 (socket says 172.21.15.40:44054) 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fd364640 1 -- 172.21.15.40:0/985987859 learned_addr learned my addr 172.21.15.40:0/985987859 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fe366640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83f80b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948172250 0 0) 0x7f83f8143d10 con 0x7f83f809e1f0 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fdb65640 1 --2- 172.21.15.40:0/985987859 >> v2:172.21.15.204:3300/0 conn(0x7f83f806d130 0x7f83f809dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.920+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83e4003530 con 0x7f83f809e1f0 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 456223803 0 0) 0x7f83e4003530 con 0x7f83f809e1f0 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 msgr2=0x7f83f80b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83f80b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 >> v2:172.21.15.204:3300/0 conn(0x7f83f806d130 msgr2=0x7f83f809dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 --2- 172.21.15.40:0/985987859 >> v2:172.21.15.204:3300/0 conn(0x7f83f806d130 0x7f83f809dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f806d510 con 0x7f83f809e1f0 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f83e8001680 con 0x7f83f809e1f0 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 >> v1:172.21.15.40:6789/0 conn(0x7f83f809e1f0 legacy=0x7f83f8144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4003b90 0x7f83e4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83e400a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e400ae80 0x7f83e400d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80ae7c0 con 0x7f83e400ae80 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80af060 con 0x7f83e4003b90 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83e400dc10 con 0x7f83f80b4e90 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e400ae80 0x7f83e400d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fdb65640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4003b90 0x7f83e4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fd364640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83e400a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 -- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 msgr2=0x7f83e400a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f80b4e90 0x7f83e400a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 -- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4003b90 msgr2=0x7f83e4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4003b90 0x7f83e4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 -- 172.21.15.40:0/985987859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e4003530 con 0x7f83e400ae80 2024-04-02T16:12:10.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fdb65640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e4003b90 0x7f83e4003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:10.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e400ae80 0x7f83e400d310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009fc0 tx=0x7f83f001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14968d857df08fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00b2070 con 0x7f83e400ae80 2024-04-02T16:12:10.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f00a5c90 con 0x7f83e400ae80 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fcb63640 1 -- 172.21.15.40:0/985987859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0009c10 con 0x7f83e400ae80 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 -- 172.21.15.40:0/985987859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e400ae80 msgr2=0x7f83e400d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/985987859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83e400ae80 0x7f83e400d310 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009fc0 tx=0x7f83f001cc80 comp rx=0 tx=0).stop 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83fe366640 1 -- 172.21.15.40:0/985987859 reap_dead start 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 -- 172.21.15.40:0/985987859 shutdown_connections 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 -- 172.21.15.40:0/985987859 >> 172.21.15.40:0/985987859 conn(0x7f83f80b8fd0 msgr2=0x7f83f80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 -- 172.21.15.40:0/985987859 shutdown_connections 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.924+0000 7f83ffdc9640 1 -- 172.21.15.40:0/985987859 wait complete. 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 Processor -- start 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- start start 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e400ae80 0x7f83f8071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f81ce400 0x7f83f81e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8144c30 con 0x7f83f81e0a40 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83f80aecf0 con 0x7f83e400ae80 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83f80ae300 con 0x7f83f81ce400 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39932/0 (socket says 172.21.15.40:39932) 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 -- 172.21.15.40:0/2797372564 learned_addr learned my addr 172.21.15.40:0/2797372564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fd364640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f81ce400 0x7f83f81e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 -- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f81ce400 msgr2=0x7f83f81e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f81ce400 0x7f83f81e0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 -- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e400ae80 msgr2=0x7f83f8071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fdb65640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e400ae80 0x7f83f8071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e400ae80 0x7f83f8071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 -- 172.21.15.40:0/2797372564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f806f9b0 con 0x7f83f81e0a40 2024-04-02T16:12:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83fe366640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83f0002240 tx=0x7f83f0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00bd070 con 0x7f83f81e0a40 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f00a5a90 con 0x7f83f81e0a40 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f80721a0 con 0x7f83f81e0a40 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00b2040 con 0x7f83f81e0a40 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f80af450 con 0x7f83f81e0a40 2024-04-02T16:12:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f80685f0 con 0x7f83f81e0a40 2024-04-02T16:12:10.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.928+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f83f00c0070 con 0x7f83f81e0a40 2024-04-02T16:12:10.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.932+0000 7f83ee7fc640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 0x7f83dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:10.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.932+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f83f01203c0 con 0x7f83f81e0a40 2024-04-02T16:12:10.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.932+0000 7f83fdb65640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 0x7f83dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:10.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.932+0000 7f83fdb65640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 0x7f83dc06a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009a60 tx=0x7f83f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:10.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:10.936+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f83f0016990 con 0x7f83f81e0a40 2024-04-02T16:12:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.152+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f83f80b82c0 con 0x7f83f81e0a40 2024-04-02T16:12:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.156+0000 7f83ee7fc640 1 -- 172.21.15.40:0/2797372564 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f83f00f0710 con 0x7f83f81e0a40 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 msgr2=0x7f83dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 0x7f83dc06a840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009a60 tx=0x7f83f4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 msgr2=0x7f83f813f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83f0002240 tx=0x7f83f0009750 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 shutdown_connections 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83f81ce400 0x7f83f81e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f83dc068380 0x7f83dc06a840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83f81e0a40 0x7f83f813f080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 --2- 172.21.15.40:0/2797372564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e400ae80 0x7f83f8071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 >> 172.21.15.40:0/2797372564 conn(0x7f83f80b8fd0 msgr2=0x7f83f8141110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 shutdown_connections 2024-04-02T16:12:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.160+0000 7f83ffdc9640 1 -- 172.21.15.40:0/2797372564 wait complete. 2024-04-02T16:12:11.176 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-02T16:12:11.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 Processor -- start 2024-04-02T16:12:11.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 -- start start 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b3500 0x7fd6f814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f80b6310 con 0x7fd6f814b390 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f80b55d0 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd7003fb640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f80b3fa0 con 0x7fd6f80b3500 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39942/0 (socket says 172.21.15.40:39942) 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 -- 172.21.15.40:0/4004367749 learned_addr learned my addr 172.21.15.40:0/4004367749 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fd996640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b3500 0x7fd6f814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 -- 172.21.15.40:0/4004367749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b3500 msgr2=0x7fd6f814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b3500 0x7fd6f814ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 -- 172.21.15.40:0/4004367749 >> v1:172.21.15.204:6789/0 conn(0x7fd6f814b390 legacy=0x7fd6f80b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:11.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.292+0000 7fd6fe197640 1 -- 172.21.15.40:0/4004367749 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f80b4150 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd6fe197640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009870 tx=0x7fd6f401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf538bcdaed8a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd6fd195640 1 -- 172.21.15.40:0/4004367749 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40b0070 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd6fd195640 1 -- 172.21.15.40:0/4004367749 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f40a4e50 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd6fd195640 1 -- 172.21.15.40:0/4004367749 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40ab820 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- 172.21.15.40:0/4004367749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b4bf0 msgr2=0x7fd6f80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009870 tx=0x7fd6f401c400 comp rx=0 tx=0).stop 2024-04-02T16:12:11.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- 172.21.15.40:0/4004367749 shutdown_connections 2024-04-02T16:12:11.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b3500 0x7fd6f814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 --2- 172.21.15.40:0/4004367749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b4bf0 0x7fd6f80b4fd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- 172.21.15.40:0/4004367749 >> 172.21.15.40:0/4004367749 conn(0x7fd6f80734b0 msgr2=0x7fd6f80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:11.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- 172.21.15.40:0/4004367749 shutdown_connections 2024-04-02T16:12:11.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- 172.21.15.40:0/4004367749 wait complete. 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 Processor -- start 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.296+0000 7fd7003fb640 1 -- start start 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b4bf0 0x7fd6f81e0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6f814b390 0x7fd6f81e3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8154c10 con 0x7fd6f80b3500 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8154d90 con 0x7fd6f814b390 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f8154f10 con 0x7fd6f80b4bf0 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39950/0 (socket says 172.21.15.40:39950) 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 -- 172.21.15.40:0/269943776 learned_addr learned my addr 172.21.15.40:0/269943776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fd996640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b4bf0 0x7fd6f81e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe998640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6f814b390 0x7fd6f81e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 -- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b4bf0 msgr2=0x7fd6f81e0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b4bf0 0x7fd6f81e0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 -- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6f814b390 msgr2=0x7fd6f81e3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6f814b390 0x7fd6f81e3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 -- 172.21.15.40:0/269943776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f81448f0 con 0x7fd6f80b3500 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fe197640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40ae040 tx=0x7fd6f40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40b0040 con 0x7fd6f80b3500 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6f40b9070 con 0x7fd6f80b3500 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f81474a0 con 0x7fd6f80b3500 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40b48b0 con 0x7fd6f80b3500 2024-04-02T16:12:11.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f8140d90 con 0x7fd6f80b3500 2024-04-02T16:12:11.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8140fd0 con 0x7fd6f80b3500 2024-04-02T16:12:11.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6f40b4a50 con 0x7fd6f80b3500 2024-04-02T16:12:11.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 0x7fd6dc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fd6f41200f0 con 0x7fd6f80b3500 2024-04-02T16:12:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.300+0000 7fd6fd996640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 0x7fd6dc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.304+0000 7fd6fd996640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 0x7fd6dc06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80bcf70 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.304+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd6f80b4150 con 0x7fd6f80b3500 2024-04-02T16:12:11.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.532+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd6f81dff00 con 0x7fd6f80b3500 2024-04-02T16:12:11.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd6eeffd640 1 -- 172.21.15.40:0/269943776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd6f4016110 con 0x7fd6f80b3500 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 msgr2=0x7fd6dc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 0x7fd6dc06a840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80bcf70 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 msgr2=0x7fd6f8148280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40ae040 tx=0x7fd6f40191a0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 shutdown_connections 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6f80b4bf0 0x7fd6f81e0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd6dc068380 0x7fd6dc06a840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6f814b390 0x7fd6f81e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 --2- 172.21.15.40:0/269943776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6f80b3500 0x7fd6f8148280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.536+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 >> 172.21.15.40:0/269943776 conn(0x7fd6f80734b0 msgr2=0x7fd6f80ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:11.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.540+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 shutdown_connections 2024-04-02T16:12:11.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.540+0000 7fd7003fb640 1 -- 172.21.15.40:0/269943776 wait complete. 2024-04-02T16:12:11.557 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-02T16:12:11.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 Processor -- start 2024-04-02T16:12:11.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 -- start start 2024-04-02T16:12:11.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe7700b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7700b6310 con 0x7fe7700b8c80 2024-04-02T16:12:11.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7700b55d0 con 0x7fe7700b3500 2024-04-02T16:12:11.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.672+0000 7fe774e16640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe77014b560 con 0x7fe7700b4bf0 2024-04-02T16:12:11.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe7700b8c80 0x7fe7701521e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44066/0 (socket says 172.21.15.40:44066) 2024-04-02T16:12:11.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76ed9d640 1 -- 172.21.15.40:0/1678353723 learned_addr learned my addr 172.21.15.40:0/1678353723 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:11.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe7700b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406030871 0 0) 0x7fe7700b6310 con 0x7fe7700b8c80 2024-04-02T16:12:11.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe75c003530 con 0x7fe7700b8c80 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1153199525 0 0) 0x7fe75c003530 con 0x7fe7700b8c80 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 msgr2=0x7fe7700b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe7700b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 >> v1:172.21.15.204:6789/0 conn(0x7fe7700b3500 legacy=0x7fe77014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7700b3fa0 con 0x7fe7700b8c80 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fe768001a00 con 0x7fe7700b8c80 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 >> v1:172.21.15.40:6789/0 conn(0x7fe7700b8c80 legacy=0x7fe7701521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe75c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe75c004810 0x7fe75c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe75c0052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe77014b560 con 0x7fe75c0039f0 2024-04-02T16:12:11.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c00bb90 con 0x7fe75c004810 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe75c00bd90 con 0x7fe7700b4bf0 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe75c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76ed9d640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe75c0052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe767fff640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe75c004810 0x7fe75c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 -- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 msgr2=0x7fe75c0052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7700b4bf0 0x7fe75c0052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 -- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe75c004810 msgr2=0x7fe75c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe75c004810 0x7fe75c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 -- 172.21.15.40:0/1678353723 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c003530 con 0x7fe75c0039f0 2024-04-02T16:12:11.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe75c003e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe758009ae0 tx=0x7fe75801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78783936584532d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580096b0 con 0x7fe75c0039f0 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7580a6450 con 0x7fe75c0039f0 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76dd9b640 1 -- 172.21.15.40:0/1678353723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580a6840 con 0x7fe75c0039f0 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 -- 172.21.15.40:0/1678353723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 msgr2=0x7fe75c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 --2- 172.21.15.40:0/1678353723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe75c003e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe758009ae0 tx=0x7fe75801ca70 comp rx=0 tx=0).stop 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe76ed9d640 1 -- 172.21.15.40:0/1678353723 reap_dead start 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 -- 172.21.15.40:0/1678353723 shutdown_connections 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 -- 172.21.15.40:0/1678353723 >> 172.21.15.40:0/1678353723 conn(0x7fe7700734b0 msgr2=0x7fe7700bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 -- 172.21.15.40:0/1678353723 shutdown_connections 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.676+0000 7fe774e16640 1 -- 172.21.15.40:0/1678353723 wait complete. 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 Processor -- start 2024-04-02T16:12:11.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- start start 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7701e4550 0x7fe7701e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe77013f800 0x7fe77013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe770154320 con 0x7fe75c0039f0 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7701544a0 con 0x7fe7701e4550 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe770154620 con 0x7fe77013f800 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39968/0 (socket says 172.21.15.40:39968) 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 -- 172.21.15.40:0/1818106773 learned_addr learned my addr 172.21.15.40:0/1818106773 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe767fff640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7701e4550 0x7fe7701e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76ed9d640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe77013f800 0x7fe77013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 -- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe77013f800 msgr2=0x7fe77013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe77013f800 0x7fe77013fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 -- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7701e4550 msgr2=0x7fe7701e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7701e4550 0x7fe7701e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 -- 172.21.15.40:0/1818106773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7701e4070 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe76e59c640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe75801c8a0 tx=0x7fe75c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580be070 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7580a7430 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe770147f30 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580096b0 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe770147c50 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.680+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7700685f0 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.684+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe7580c1070 con 0x7fe75c0039f0 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.684+0000 7fe766ffd640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 0x7fe75006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:11.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.684+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fe758120320 con 0x7fe75c0039f0 2024-04-02T16:12:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.684+0000 7fe767fff640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 0x7fe75006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:11.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.684+0000 7fe767fff640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 0x7fe75006a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe770145450 tx=0x7fe760009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:11.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.688+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fe7580ac020 con 0x7fe75c0039f0 2024-04-02T16:12:11.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.904+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe7700bb0d0 con 0x7fe75c0039f0 2024-04-02T16:12:11.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.908+0000 7fe766ffd640 1 -- 172.21.15.40:0/1818106773 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fe758016780 con 0x7fe75c0039f0 2024-04-02T16:12:11.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 msgr2=0x7fe75006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 0x7fe75006a840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe770145450 tx=0x7fe760009040 comp rx=0 tx=0).stop 2024-04-02T16:12:11.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 msgr2=0x7fe77013d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe75801c8a0 tx=0x7fe75c005840 comp rx=0 tx=0).stop 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 shutdown_connections 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe77013f800 0x7fe77013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe750068380 0x7fe75006a840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7701e4550 0x7fe7701e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 --2- 172.21.15.40:0/1818106773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe75c0039f0 0x7fe77013d970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 >> 172.21.15.40:0/1818106773 conn(0x7fe7700734b0 msgr2=0x7fe7700ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 shutdown_connections 2024-04-02T16:12:11.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:11.912+0000 7fe774e16640 1 -- 172.21.15.40:0/1818106773 wait complete. 2024-04-02T16:12:11.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-02T16:12:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.036+0000 7f2abd66a640 1 Processor -- start 2024-04-02T16:12:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.036+0000 7f2abd66a640 1 -- start start 2024-04-02T16:12:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2abd66a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ab80b4bf0 0x7f2ab80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2abd66a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80b6310 con 0x7f2ab80b8c80 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2abd66a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80b55d0 con 0x7f2ab80b3500 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2abd66a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2ab814b560 con 0x7f2ab80b4bf0 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab6ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ab80b4bf0 0x7f2ab80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab77fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ab80b8c80 0x7f2ab81521e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44078/0 (socket says 172.21.15.40:44078) 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab77fe640 1 -- 172.21.15.40:0/1084442844 learned_addr learned my addr 172.21.15.40:0/1084442844 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab67fc640 1 -- 172.21.15.40:0/1084442844 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414448858 0 0) 0x7f2ab80b6310 con 0x7f2ab80b8c80 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab67fc640 1 -- 172.21.15.40:0/1084442844 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa4003530 con 0x7f2ab80b8c80 2024-04-02T16:12:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1084442844 >> v1:172.21.15.204:6789/0 conn(0x7f2ab80b3500 legacy=0x7f2ab814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1084442844 >> v1:172.21.15.40:6789/0 conn(0x7f2ab80b8c80 legacy=0x7f2ab81521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1084442844 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab80b3fa0 con 0x7f2ab80b4bf0 2024-04-02T16:12:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab6ffd640 1 --2- 172.21.15.40:0/1084442844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b4bf0 0x7f2ab80b4fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80097c0 tx=0x7f2aa801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de9455983ee8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab67fc640 1 -- 172.21.15.40:0/1084442844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80b0070 con 0x7f2ab80b4bf0 2024-04-02T16:12:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab67fc640 1 -- 172.21.15.40:0/1084442844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa80ab600 con 0x7f2ab80b4bf0 2024-04-02T16:12:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.040+0000 7f2ab67fc640 1 -- 172.21.15.40:0/1084442844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80ab9d0 con 0x7f2ab80b4bf0 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1084442844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b4bf0 msgr2=0x7f2ab80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1084442844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b4bf0 0x7f2ab80b4fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80097c0 tx=0x7f2aa801c380 comp rx=0 tx=0).stop 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1084442844 shutdown_connections 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1084442844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b4bf0 0x7f2ab80b4fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1084442844 >> 172.21.15.40:0/1084442844 conn(0x7f2ab80734b0 msgr2=0x7f2ab80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1084442844 shutdown_connections 2024-04-02T16:12:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1084442844 wait complete. 2024-04-02T16:12:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 Processor -- start 2024-04-02T16:12:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- start start 2024-04-02T16:12:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ab80b8c80 0x7f2ab8130990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8130ed0 0x7f2ab813b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8156740 con 0x7f2ab80b3500 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab81568c0 con 0x7f2ab8130ed0 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab8156a40 con 0x7f2ab80b8c80 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39994/0 (socket says 172.21.15.40:39994) 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1597515711 learned_addr learned my addr 172.21.15.40:0/1597515711 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ab80b8c80 msgr2=0x7f2ab8130990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ab80b8c80 0x7f2ab8130990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8130ed0 msgr2=0x7f2ab813b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8130ed0 0x7f2ab813b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 -- 172.21.15.40:0/1597515711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab8145c00 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2ab6ffd640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ae040 tx=0x7f2aa8019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80b0040 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa80b9070 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80aa970 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab81301f0 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab8146150 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.044+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab80685f0 con 0x7f2ab80b3500 2024-04-02T16:12:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.048+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f2aa80aab10 con 0x7f2ab80b3500 2024-04-02T16:12:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.048+0000 7f2aaf7fe640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 0x7f2a9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.048+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f2aa81206d0 con 0x7f2ab80b3500 2024-04-02T16:12:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.048+0000 7f2aaffff640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 0x7f2a9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.048+0000 7f2aaffff640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 0x7f2a9806a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000f70 tx=0x7f2aa00079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.052+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2aa8016090 con 0x7f2ab80b3500 2024-04-02T16:12:12.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.264+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2ab80b6da0 con 0x7f2ab80b3500 2024-04-02T16:12:12.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.268+0000 7f2aaf7fe640 1 -- 172.21.15.40:0/1597515711 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f2aa80f0fd0 con 0x7f2ab80b3500 2024-04-02T16:12:12.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 msgr2=0x7f2a9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 0x7f2a9806a840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000f70 tx=0x7f2aa00079c0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 msgr2=0x7f2ab81e40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ae040 tx=0x7f2aa8019120 comp rx=0 tx=0).stop 2024-04-02T16:12:12.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 shutdown_connections 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ab80b8c80 0x7f2ab8130990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2a98068380 0x7f2a9806a840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab8130ed0 0x7f2ab813b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 --2- 172.21.15.40:0/1597515711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab80b3500 0x7f2ab81e40f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 >> 172.21.15.40:0/1597515711 conn(0x7f2ab80734b0 msgr2=0x7f2ab8135710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 shutdown_connections 2024-04-02T16:12:12.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.272+0000 7f2abd66a640 1 -- 172.21.15.40:0/1597515711 wait complete. 2024-04-02T16:12:12.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:12.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.404+0000 7f7435e5e640 1 Processor -- start 2024-04-02T16:12:12.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.404+0000 7f7435e5e640 1 -- start start 2024-04-02T16:12:12.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f7435e5e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7430073230 0x7f74300b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f7435e5e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300b19e0 con 0x7f7430148e50 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f7435e5e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74300b05c0 con 0x7f74300b23d0 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f7435e5e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f74300b08a0 con 0x7f7430073230 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7430148e50 0x7f7430149280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44080/0 (socket says 172.21.15.40:44080) 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742ffff640 1 -- 172.21.15.40:0/539702207 learned_addr learned my addr 172.21.15.40:0/539702207 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742effd640 1 --2- 172.21.15.40:0/539702207 >> v2:172.21.15.204:3300/0 conn(0x7f7430073230 0x7f74300b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813641024 0 0) 0x7f74300b19e0 con 0x7f7430148e50 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f741c007940 con 0x7f7430148e50 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266783145 0 0) 0x7f741c007940 con 0x7f7430148e50 2024-04-02T16:12:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 >> v2:172.21.15.204:3300/0 conn(0x7f7430073230 msgr2=0x7f74300b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 --2- 172.21.15.40:0/539702207 >> v2:172.21.15.204:3300/0 conn(0x7f7430073230 0x7f74300b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 >> v1:172.21.15.204:6789/0 conn(0x7f74300b23d0 legacy=0x7f743009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74300b0b80 con 0x7f7430148e50 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f7428001680 con 0x7f7430148e50 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 >> v1:172.21.15.40:6789/0 conn(0x7f7430148e50 legacy=0x7f7430149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 0x7f741c0081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f741c008c20 0x7f741c0090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c0095e0 0x7f741c00fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74300b08a0 con 0x7f741c007dd0 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f741c010580 con 0x7f741c0095e0 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f741c010780 con 0x7f741c008c20 2024-04-02T16:12:12.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 0x7f741c0081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742ffff640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c0095e0 0x7f741c00fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 -- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f741c008c20 msgr2=0x7f741c0090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f741c008c20 0x7f741c0090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 -- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c0095e0 msgr2=0x7f741c00fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c0095e0 0x7f741c00fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 -- 172.21.15.40:0/539702207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c007940 con 0x7f741c007dd0 2024-04-02T16:12:12.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742ffff640 1 -- 172.21.15.40:0/539702207 reap_dead start 2024-04-02T16:12:12.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742f7fe640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 0x7f741c0081f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7420001e70 tx=0x7f742001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5325f8b2792bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200af070 con 0x7f741c007dd0 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7420003b80 con 0x7f741c007dd0 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.408+0000 7f742e7fc640 1 -- 172.21.15.40:0/539702207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420001a90 con 0x7f741c007dd0 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 msgr2=0x7f741c0081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 0x7f741c0081f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7420001e70 tx=0x7f742001c760 comp rx=0 tx=0).stop 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/539702207 shutdown_connections 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 --2- 172.21.15.40:0/539702207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f741c007dd0 0x7f741c0081f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/539702207 >> 172.21.15.40:0/539702207 conn(0x7f74300b8fd0 msgr2=0x7f74300b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/539702207 shutdown_connections 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/539702207 wait complete. 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 Processor -- start 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- start start 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c007dd0 0x7f74301d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f743009add0 0x7f743009b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f74301458e0 con 0x7f74301dd700 2024-04-02T16:12:12.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7430145760 con 0x7f741c007dd0 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7430145a60 con 0x7f743009add0 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40014/0 (socket says 172.21.15.40:40014) 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 -- 172.21.15.40:0/1022145502 learned_addr learned my addr 172.21.15.40:0/1022145502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742f7fe640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c007dd0 0x7f74301d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742effd640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f743009add0 0x7f743009b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 -- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f743009add0 msgr2=0x7f743009b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f743009add0 0x7f743009b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 -- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c007dd0 msgr2=0x7f74301d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c007dd0 0x7f74301d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 -- 172.21.15.40:0/1022145502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743009b8e0 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f742ffff640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74300b3cb0 tx=0x7f742800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428012590 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7428018070 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74301cfec0 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428012990 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301e0680 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.412+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74300685f0 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.416+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f7428012b30 con 0x7f74301dd700 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.416+0000 7f7427fff640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 0x7f741006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.416+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f74280861b0 con 0x7f74301dd700 2024-04-02T16:12:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.416+0000 7f742f7fe640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 0x7f741006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.416+0000 7f742f7fe640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 0x7f741006a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74200ac000 tx=0x7f7420001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.420+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7428056500 con 0x7f74301dd700 2024-04-02T16:12:12.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.576+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74301488e0 con 0x7f74301dd700 2024-04-02T16:12:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.576+0000 7f7427fff640 1 -- 172.21.15.40:0/1022145502 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+12843 (secure 0 0 0) 0x7f742805a800 con 0x7f74301dd700 2024-04-02T16:12:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:12.579 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:09.650601+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 msgr2=0x7f741006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 0x7f741006a840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f74200ac000 tx=0x7f7420001520 comp rx=0 tx=0).stop 2024-04-02T16:12:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 msgr2=0x7f74301ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74300b3cb0 tx=0x7f742800e960 comp rx=0 tx=0).stop 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 shutdown_connections 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f743009add0 0x7f743009b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7410068380 0x7f741006a840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74301dd700 0x7f74301ddb80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 --2- 172.21.15.40:0/1022145502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f741c007dd0 0x7f74301d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 >> 172.21.15.40:0/1022145502 conn(0x7f74300b8fd0 msgr2=0x7f743006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 shutdown_connections 2024-04-02T16:12:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.580+0000 7f7435e5e640 1 -- 172.21.15.40:0/1022145502 wait complete. 2024-04-02T16:12:12.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:12:12.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:12:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.716+0000 7f0bbb98b640 1 Processor -- start 2024-04-02T16:12:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.716+0000 7f0bbb98b640 1 -- start start 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bbb98b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0bb414b040 0x7f0bb414b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bbb98b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bb40b9580 0x7f0bb414b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bbb98b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb40b7e00 con 0x7f0bb40b3780 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bbb98b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb40b7bb0 con 0x7f0bb40b9580 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bbb98b640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb40bc580 con 0x7f0bb414b040 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bb40b9580 0x7f0bb414b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb9f28640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bb40b3780 0x7f0bb4141740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44086/0 (socket says 172.21.15.40:44086) 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb9f28640 1 -- 172.21.15.40:0/894837657 learned_addr learned my addr 172.21.15.40:0/894837657 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb9727640 1 --2- 172.21.15.40:0/894837657 >> v2:172.21.15.204:3300/0 conn(0x7f0bb414b040 0x7f0bb414b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 -- 172.21.15.40:0/894837657 >> v2:172.21.15.204:3300/0 conn(0x7f0bb414b040 msgr2=0x7f0bb414b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 --2- 172.21.15.40:0/894837657 >> v2:172.21.15.204:3300/0 conn(0x7f0bb414b040 0x7f0bb414b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 -- 172.21.15.40:0/894837657 >> v1:172.21.15.40:6789/0 conn(0x7f0bb40b3780 legacy=0x7f0bb4141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 -- 172.21.15.40:0/894837657 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb40bce00 con 0x7f0bb40b9580 2024-04-02T16:12:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0babfff640 1 -- 172.21.15.40:0/894837657 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292445001 0 0) 0x7f0bb40b7e00 con 0x7f0bb40b3780 2024-04-02T16:12:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0bb8f26640 1 --2- 172.21.15.40:0/894837657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b9580 0x7f0bb414b960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0bb402d560 tx=0x7f0ba401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7fb22e411912d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0babfff640 1 -- 172.21.15.40:0/894837657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40b0070 con 0x7f0bb40b9580 2024-04-02T16:12:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0babfff640 1 -- 172.21.15.40:0/894837657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba40ab600 con 0x7f0bb40b9580 2024-04-02T16:12:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.720+0000 7f0babfff640 1 -- 172.21.15.40:0/894837657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40ab9d0 con 0x7f0bb40b9580 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/894837657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b9580 msgr2=0x7f0bb414b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/894837657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b9580 0x7f0bb414b960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0bb402d560 tx=0x7f0ba401c360 comp rx=0 tx=0).stop 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/894837657 shutdown_connections 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/894837657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b9580 0x7f0bb414b960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/894837657 >> v2:172.21.15.204:3300/0 conn(0x7f0bb414b040 0x7f0bb414b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/894837657 >> 172.21.15.40:0/894837657 conn(0x7f0bb40bd230 msgr2=0x7f0bb40bd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/894837657 shutdown_connections 2024-04-02T16:12:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/894837657 wait complete. 2024-04-02T16:12:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 Processor -- start 2024-04-02T16:12:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- start start 2024-04-02T16:12:12.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40b9580 0x7f0bb40a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb414b040 0x7f0bb40a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4154b60 con 0x7f0bb40b3780 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4142060 con 0x7f0bb40b9580 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb4154ce0 con 0x7f0bb414b040 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40036/0 (socket says 172.21.15.40:40036) 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 -- 172.21.15.40:0/3331303594 learned_addr learned my addr 172.21.15.40:0/3331303594 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 -- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb414b040 msgr2=0x7f0bb40a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb414b040 0x7f0bb40a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 -- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40b9580 msgr2=0x7f0bb40a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40b9580 0x7f0bb40a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 -- 172.21.15.40:0/3331303594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb4142d00 con 0x7f0bb40b3780 2024-04-02T16:12:12.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bb9727640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0007e90 tx=0x7f0bb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0014420 con 0x7f0bb40b3780 2024-04-02T16:12:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb0019070 con 0x7f0bb40b3780 2024-04-02T16:12:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0014800 con 0x7f0bb40b3780 2024-04-02T16:12:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb40a0160 con 0x7f0bb40b3780 2024-04-02T16:12:12.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.724+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41e04d0 con 0x7f0bb40b3780 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb40685f0 con 0x7f0bb40b3780 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f0bb00149a0 con 0x7f0bb40b3780 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0ba9ffb640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 0x7f0b9806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f0bb0086f10 con 0x7f0bb40b3780 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0bb8f26640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 0x7f0b9806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:12.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.728+0000 7f0bb8f26640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 0x7f0b9806a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ad000 tx=0x7f0ba4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:12.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.732+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f0bb0057810 con 0x7f0bb40b3780 2024-04-02T16:12:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.896+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0bb4142a20 con 0x7f0bb40b3780 2024-04-02T16:12:12.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.896+0000 7f0ba9ffb640 1 -- 172.21.15.40:0/3331303594 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f0bb0057110 con 0x7f0bb40b3780 2024-04-02T16:12:12.899 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 msgr2=0x7f0b9806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 0x7f0b9806a840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ad000 tx=0x7f0ba4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 msgr2=0x7f0bb413bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0007e90 tx=0x7f0bb000e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 shutdown_connections 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0bb414b040 0x7f0bb40a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0b98068380 0x7f0b9806a840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb40b9580 0x7f0bb40a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 --2- 172.21.15.40:0/3331303594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb40b3780 0x7f0bb413bfe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:12.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 >> 172.21.15.40:0/3331303594 conn(0x7f0bb40bd230 msgr2=0x7f0bb40b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:12.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 shutdown_connections 2024-04-02T16:12:12.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:12.900+0000 7f0bbb98b640 1 -- 172.21.15.40:0/3331303594 wait complete. 2024-04-02T16:12:12.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T16:12:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 Processor -- start 2024-04-02T16:12:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 -- start start 2024-04-02T16:12:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb406e070 0x7fbdb4141440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbdb40b5860 0x7fbdb4141980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb40bb850 con 0x7fbdb40bb470 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb406d0d0 con 0x7fbdb40b5860 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdba8cc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb40b6300 con 0x7fbdb406e070 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbdb40b5860 0x7fbdb4141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbdb40b5860 0x7fbdb4141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40038/0 (socket says 172.21.15.40:40038) 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 -- 172.21.15.40:0/3255409206 learned_addr learned my addr 172.21.15.40:0/3255409206 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 -- 172.21.15.40:0/3255409206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb406e070 msgr2=0x7fbdb4141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 --2- 172.21.15.40:0/3255409206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb406e070 0x7fbdb4141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 -- 172.21.15.40:0/3255409206 >> v1:172.21.15.204:6789/0 conn(0x7fbdb40bb470 legacy=0x7fbdb406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.032+0000 7fbdb8e69640 1 -- 172.21.15.40:0/3255409206 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb40b6b90 con 0x7fbdb40b5860 2024-04-02T16:12:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb8e69640 1 --2- 172.21.15.40:0/3255409206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb4141980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009870 tx=0x7fbda401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43dc7df3726f1c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb2ffd640 1 -- 172.21.15.40:0/3255409206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40ab650 con 0x7fbdb40b5860 2024-04-02T16:12:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb2ffd640 1 -- 172.21.15.40:0/3255409206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda40b0070 con 0x7fbdb40b5860 2024-04-02T16:12:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb2ffd640 1 -- 172.21.15.40:0/3255409206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40aba10 con 0x7fbdb40b5860 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/3255409206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 msgr2=0x7fbdb4141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/3255409206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb4141980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009870 tx=0x7fbda401c490 comp rx=0 tx=0).stop 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/3255409206 shutdown_connections 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/3255409206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb4141980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/3255409206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb406e070 0x7fbdb4141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/3255409206 >> 172.21.15.40:0/3255409206 conn(0x7fbdb402c5d0 msgr2=0x7fbdb402c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/3255409206 shutdown_connections 2024-04-02T16:12:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/3255409206 wait complete. 2024-04-02T16:12:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 Processor -- start 2024-04-02T16:12:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- start start 2024-04-02T16:12:13.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdb406e070 0x7fbdb4142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb40bb470 0x7fbdb40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb4154860 con 0x7fbdb40b5860 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb4073420 con 0x7fbdb406e070 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb41549e0 con 0x7fbdb40bb470 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40054/0 (socket says 172.21.15.40:40054) 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 -- 172.21.15.40:0/1904006549 learned_addr learned my addr 172.21.15.40:0/1904006549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb8e69640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb40bb470 0x7fbdb40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb3fff640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdb406e070 0x7fbdb4142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 -- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb40bb470 msgr2=0x7fbdb40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb40bb470 0x7fbdb40b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 -- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdb406e070 msgr2=0x7fbdb4142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdb406e070 0x7fbdb4142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 -- 172.21.15.40:0/1904006549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb41420d0 con 0x7fbdb40b5860 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb37fe640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b4b0 tx=0x7fbd9c00b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c014070 con 0x7fbdb40b5860 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c010610 con 0x7fbdb40b5860 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c010a00 con 0x7fbdb40b5860 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb4142370 con 0x7fbdb40b5860 2024-04-02T16:12:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.036+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb4143330 con 0x7fbdb40b5860 2024-04-02T16:12:13.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4068d00 con 0x7fbdb40b5860 2024-04-02T16:12:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fbd9c017070 con 0x7fbdb40b5860 2024-04-02T16:12:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdb0ff9640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fbd9c086080 con 0x7fbdb40b5860 2024-04-02T16:12:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdb3fff640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.040+0000 7fbdb3fff640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 0x7fbd9406a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbdac007bf0 tx=0x7fbdac007240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.044+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fbd9c056990 con 0x7fbdb40b5860 2024-04-02T16:12:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.200+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbdb4140050 con 0x7fbdb40b5860 2024-04-02T16:12:13.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.200+0000 7fbdb0ff9640 1 -- 172.21.15.40:0/1904006549 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v25) ==== 84+0+65 (secure 0 0 0) 0x7fbd9c056290 con 0x7fbdb40b5860 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:13.204 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-04-02T16:12:13.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 msgr2=0x7fbd9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 0x7fbd9406a840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbdac007bf0 tx=0x7fbdac007240 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 msgr2=0x7fbdb40b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b4b0 tx=0x7fbd9c00b970 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 shutdown_connections 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdb40bb470 0x7fbdb40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fbd94068380 0x7fbd9406a840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdb40b5860 0x7fbdb40b0920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 --2- 172.21.15.40:0/1904006549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdb406e070 0x7fbdb4142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 >> 172.21.15.40:0/1904006549 conn(0x7fbdb402c5d0 msgr2=0x7fbdb4145710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 shutdown_connections 2024-04-02T16:12:13.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.204+0000 7fbdba8cc640 1 -- 172.21.15.40:0/1904006549 wait complete. 2024-04-02T16:12:13.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T16:12:13.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T16:12:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 Processor -- start 2024-04-02T16:12:13.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 -- start start 2024-04-02T16:12:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a006d830 0x7f57a00b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57a00b5320 0x7f57a0072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a00b4ec0 con 0x7f57a00730a0 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f57a00b2c40 con 0x7f57a00b5320 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f57a512a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57a00b2560 con 0x7f57a006d830 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57a00b5320 0x7f57a0072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57a00b5320 0x7f57a0072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40066/0 (socket says 172.21.15.40:40066) 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 -- 172.21.15.40:0/786442828 learned_addr learned my addr 172.21.15.40:0/786442828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579ed9d640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a006d830 0x7f57a00b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 -- 172.21.15.40:0/786442828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a006d830 msgr2=0x7f57a00b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a006d830 0x7f57a00b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 -- 172.21.15.40:0/786442828 >> v1:172.21.15.204:6789/0 conn(0x7f57a00730a0 legacy=0x7f57a00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.344+0000 7f579e59c640 1 -- 172.21.15.40:0/786442828 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a006dc10 con 0x7f57a00b5320 2024-04-02T16:12:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f579e59c640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a0072b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a164778310e94d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f579dd9b640 1 -- 172.21.15.40:0/786442828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900b2070 con 0x7f57a00b5320 2024-04-02T16:12:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f579dd9b640 1 -- 172.21.15.40:0/786442828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57900ad640 con 0x7f57a00b5320 2024-04-02T16:12:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f579dd9b640 1 -- 172.21.15.40:0/786442828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57900ada10 con 0x7f57a00b5320 2024-04-02T16:12:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- 172.21.15.40:0/786442828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 msgr2=0x7f57a0072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a0072b60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579001c560 comp rx=0 tx=0).stop 2024-04-02T16:12:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- 172.21.15.40:0/786442828 shutdown_connections 2024-04-02T16:12:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a0072b60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 --2- 172.21.15.40:0/786442828 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a006d830 0x7f57a00b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- 172.21.15.40:0/786442828 >> 172.21.15.40:0/786442828 conn(0x7f57a00b5890 msgr2=0x7f57a00b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- 172.21.15.40:0/786442828 shutdown_connections 2024-04-02T16:12:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- 172.21.15.40:0/786442828 wait complete. 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 Processor -- start 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.348+0000 7f57a512a640 1 -- start start 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a006d830 0x7f57a013fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a00730a0 0x7f57a013e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b2f00 con 0x7f57a00b5320 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b17d0 con 0x7f57a006d830 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57a00b27b0 con 0x7f57a00730a0 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40078/0 (socket says 172.21.15.40:40078) 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 -- 172.21.15.40:0/2517059435 learned_addr learned my addr 172.21.15.40:0/2517059435 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579ed9d640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a006d830 0x7f57a013fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579e59c640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a00730a0 0x7f57a013e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 -- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a00730a0 msgr2=0x7f57a013e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a00730a0 0x7f57a013e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 -- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a006d830 msgr2=0x7f57a013fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a006d830 0x7f57a013fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 -- 172.21.15.40:0/2517059435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a0140400 con 0x7f57a00b5320 2024-04-02T16:12:13.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579f59e640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f579800c710 tx=0x7f579800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798016020 con 0x7f57a00b5320 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798006d60 con 0x7f57a00b5320 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a00707a0 con 0x7f57a00b5320 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798003bc0 con 0x7f57a00b5320 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a00725c0 con 0x7f57a00b5320 2024-04-02T16:12:13.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a00685f0 con 0x7f57a00b5320 2024-04-02T16:12:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f579800a850 con 0x7f57a00b5320 2024-04-02T16:12:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 0x7f578c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f57980863b0 con 0x7f57a00b5320 2024-04-02T16:12:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.352+0000 7f579ed9d640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 0x7f578c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.356+0000 7f579ed9d640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 0x7f578c06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f579400b9d0 tx=0x7f5794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.356+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f5798020020 con 0x7f57a00b5320 2024-04-02T16:12:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.520+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f57a00b7680 con 0x7f57a00b5320 2024-04-02T16:12:13.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.676+0000 7f57877fe640 1 -- 172.21.15.40:0/2517059435 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v26) ==== 138+0+0 (secure 0 0 0) 0x7f5798056cb0 con 0x7f57a00b5320 2024-04-02T16:12:13.678 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-04-02T16:12:13.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 msgr2=0x7f578c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 0x7f578c06a840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f579400b9d0 tx=0x7f5794008040 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 msgr2=0x7f57a01e18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f579800c710 tx=0x7f579800cbd0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 shutdown_connections 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57a00730a0 0x7f57a013e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f578c068380 0x7f578c06a840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57a00b5320 0x7f57a01e18e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 --2- 172.21.15.40:0/2517059435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a006d830 0x7f57a013fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 >> 172.21.15.40:0/2517059435 conn(0x7f57a00b5890 msgr2=0x7f57a00b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 shutdown_connections 2024-04-02T16:12:13.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.680+0000 7f57a512a640 1 -- 172.21.15.40:0/2517059435 wait complete. 2024-04-02T16:12:13.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T16:12:13.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.840+0000 7fb4f32f4640 1 Processor -- start 2024-04-02T16:12:13.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.840+0000 7fb4f32f4640 1 -- start start 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f32f4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4ec147c50 0x7fb4ec1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f32f4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4ec148a90 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f32f4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4ec0ae830 con 0x7fb4ec148620 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f32f4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb4ec0b2b50 con 0x7fb4ec147c50 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1090640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb4ec06c8a0 0x7fb4ec0af150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44092/0 (socket says 172.21.15.40:44092) 2024-04-02T16:12:13.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1090640 1 -- 172.21.15.40:0/4136111822 learned_addr learned my addr 172.21.15.40:0/4136111822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f088f640 1 --2- 172.21.15.40:0/4136111822 >> v2:172.21.15.204:3300/0 conn(0x7fb4ec147c50 0x7fb4ec1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461334610 0 0) 0x7fb4ec148a90 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4d4003530 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2941975250 0 0) 0x7fb4d4003530 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 >> v2:172.21.15.204:3300/0 conn(0x7fb4ec147c50 msgr2=0x7fb4ec1480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 --2- 172.21.15.40:0/4136111822 >> v2:172.21.15.204:3300/0 conn(0x7fb4ec147c50 0x7fb4ec1480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 >> v1:172.21.15.204:6789/0 conn(0x7fb4ec148620 legacy=0x7fb4ec14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec06cc80 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fb4dc001680 con 0x7fb4ec06c8a0 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 >> v1:172.21.15.40:6789/0 conn(0x7fb4ec06c8a0 legacy=0x7fb4ec0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40039c0 0x7fb4d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4004810 0x7fb4d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 0x7fb4d400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0b2b50 con 0x7fb4d4005220 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d400e360 con 0x7fb4d4004810 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d400e560 con 0x7fb4d40039c0 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 0x7fb4d400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1090640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40039c0 0x7fb4d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f088f640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4004810 0x7fb4d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 -- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40039c0 msgr2=0x7fb4d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40039c0 0x7fb4d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 -- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4004810 msgr2=0x7fb4d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d4004810 0x7fb4d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.844+0000 7fb4f1891640 1 -- 172.21.15.40:0/4136111822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d4003530 con 0x7fb4d4005220 2024-04-02T16:12:13.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f1891640 1 -- 172.21.15.40:0/4136111822 reap_dead start 2024-04-02T16:12:13.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f1891640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 0x7fb4d400dbe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4001780 tx=0x7fb4e401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc5c70806407df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40013a0 con 0x7fb4d4005220 2024-04-02T16:12:13.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e40052f0 con 0x7fb4d4005220 2024-04-02T16:12:13.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4ebfff640 1 -- 172.21.15.40:0/4136111822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4003840 con 0x7fb4d4005220 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 msgr2=0x7fb4d400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 0x7fb4d400dbe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4001780 tx=0x7fb4e401c420 comp rx=0 tx=0).stop 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- 172.21.15.40:0/4136111822 shutdown_connections 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/4136111822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4d4005220 0x7fb4d400dbe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- 172.21.15.40:0/4136111822 >> 172.21.15.40:0/4136111822 conn(0x7fb4ec0b8fd0 msgr2=0x7fb4ec0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- 172.21.15.40:0/4136111822 shutdown_connections 2024-04-02T16:12:13.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- 172.21.15.40:0/4136111822 wait complete. 2024-04-02T16:12:13.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 Processor -- start 2024-04-02T16:12:13.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.848+0000 7fb4f32f4640 1 -- start start 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d4005220 0x7fb4ec1dfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec144f20 0x7fb4ec145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0aead0 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ec0b0300 con 0x7fb4ec144f20 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb4ec0b3130 con 0x7fb4d4005220 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40098/0 (socket says 172.21.15.40:40098) 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 -- 172.21.15.40:0/1829985379 learned_addr learned my addr 172.21.15.40:0/1829985379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f1891640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec144f20 0x7fb4ec145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f1090640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d4005220 0x7fb4ec1dfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 -- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d4005220 msgr2=0x7fb4ec1dfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d4005220 0x7fb4ec1dfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 -- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec144f20 msgr2=0x7fb4ec145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec144f20 0x7fb4ec145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 -- 172.21.15.40:0/1829985379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ec072d30 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f088f640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0011e30 tx=0x7fb4e000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0015630 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e00157d0 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0015bc0 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ec072a90 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ec126310 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4ec0685f0 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb4e001d400 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 0x7fb4d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:13.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.852+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5227+0+0 (secure 0 0 0) 0x7fb4e0088060 con 0x7fb4ec0b79d0 2024-04-02T16:12:13.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.856+0000 7fb4f1090640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 0x7fb4d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:13.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.856+0000 7fb4f1090640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 0x7fb4d806a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc001ce0 tx=0x7fb4dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:13.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:13.856+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb4e0019030 con 0x7fb4ec0b79d0 2024-04-02T16:12:14.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.012+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb4ec1480d0 con 0x7fb4ec0b79d0 2024-04-02T16:12:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.688+0000 7fb4e9ffb640 1 -- 172.21.15.40:0/1829985379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v27) ==== 170+0+0 (secure 0 0 0) 0x7fb4e0010040 con 0x7fb4ec0b79d0 2024-04-02T16:12:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-04-02T16:12:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 msgr2=0x7fb4d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 0x7fb4d806a840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc001ce0 tx=0x7fb4dc009040 comp rx=0 tx=0).stop 2024-04-02T16:12:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 msgr2=0x7fb4ec125020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0011e30 tx=0x7fb4e000f580 comp rx=0 tx=0).stop 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 shutdown_connections 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb4d8068380 0x7fb4d806a840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec144f20 0x7fb4ec145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4ec0b79d0 0x7fb4ec125020 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 --2- 172.21.15.40:0/1829985379 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d4005220 0x7fb4ec1dfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 >> 172.21.15.40:0/1829985379 conn(0x7fb4ec0b8fd0 msgr2=0x7fb4ec0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 shutdown_connections 2024-04-02T16:12:14.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.692+0000 7fb4f32f4640 1 -- 172.21.15.40:0/1829985379 wait complete. 2024-04-02T16:12:14.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T16:12:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 Processor -- start 2024-04-02T16:12:14.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 -- start start 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f59c40b2780 0x7f59c41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c40b0a10 con 0x7f59c40b1c10 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c40b3210 con 0x7f59c40ae080 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59ca002640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f59c406db10 con 0x7f59c40b2780 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f59c40b1c10 0x7f59c40b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44102/0 (socket says 172.21.15.40:44102) 2024-04-02T16:12:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 -- 172.21.15.40:0/2514627218 learned_addr learned my addr 172.21.15.40:0/2514627218 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c3fff640 1 --2- 172.21.15.40:0/2514627218 >> v2:172.21.15.204:3300/0 conn(0x7f59c40b2780 0x7f59c41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545504970 0 0) 0x7f59c40b0a10 con 0x7f59c40b1c10 2024-04-02T16:12:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b0003530 con 0x7f59c40b1c10 2024-04-02T16:12:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 662301858 0 0) 0x7f59b0003530 con 0x7f59c40b1c10 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 >> v2:172.21.15.204:3300/0 conn(0x7f59c40b2780 msgr2=0x7f59c41532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 --2- 172.21.15.40:0/2514627218 >> v2:172.21.15.204:3300/0 conn(0x7f59c40b2780 0x7f59c41532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 >> v1:172.21.15.204:6789/0 conn(0x7f59c40ae080 legacy=0x7f59c40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c40ae460 con 0x7f59c40b1c10 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f59ac001680 con 0x7f59c40b1c10 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 >> v1:172.21.15.40:6789/0 conn(0x7f59c40b1c10 legacy=0x7f59c40b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b00039c0 0x7f59b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59b00051d0 0x7f59b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59c406db10 con 0x7f59b0004810 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59b000c070 con 0x7f59b00039c0 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59b000c270 con 0x7f59b00051d0 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c37fe640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b00039c0 0x7f59b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c3fff640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59b00051d0 0x7f59b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 -- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59b00051d0 msgr2=0x7f59b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59b00051d0 0x7f59b000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 -- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b00039c0 msgr2=0x7f59b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b00039c0 0x7f59b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c2ffd640 1 -- 172.21.15.40:0/2514627218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b0003530 con 0x7f59b0004810 2024-04-02T16:12:14.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.852+0000 7f59c3fff640 1 -- 172.21.15.40:0/2514627218 reap_dead start 2024-04-02T16:12:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59b0004c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c1c0 tx=0x7f59ac01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e5493f4ac50d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac007d00 con 0x7f59b0004810 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac005b00 con 0x7f59b0004810 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59c27fc640 1 -- 172.21.15.40:0/2514627218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac007070 con 0x7f59b0004810 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 -- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 msgr2=0x7f59b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59b0004c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00c1c0 tx=0x7f59ac01f350 comp rx=0 tx=0).stop 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 -- 172.21.15.40:0/2514627218 shutdown_connections 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 --2- 172.21.15.40:0/2514627218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59b0004c90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 -- 172.21.15.40:0/2514627218 >> 172.21.15.40:0/2514627218 conn(0x7f59c40b8fd0 msgr2=0x7f59c40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 -- 172.21.15.40:0/2514627218 shutdown_connections 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.856+0000 7f59ca002640 1 -- 172.21.15.40:0/2514627218 wait complete. 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 Processor -- start 2024-04-02T16:12:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- start start 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59c41d0f10 0x7f59c409bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c41d2a10 0x7f59c41c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40afde0 con 0x7f59b0004810 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40b2dc0 con 0x7f59c41d2a10 2024-04-02T16:12:14.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59c40ae780 con 0x7f59c41d0f10 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40122/0 (socket says 172.21.15.40:40122) 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 -- 172.21.15.40:0/2555561799 learned_addr learned my addr 172.21.15.40:0/2555561799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59c41d0f10 0x7f59c409bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c3fff640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c41d2a10 0x7f59c41c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 -- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59c41d0f10 msgr2=0x7f59c409bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59c41d0f10 0x7f59c409bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 -- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c41d2a10 msgr2=0x7f59c41c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c41d2a10 0x7f59c41c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 -- 172.21.15.40:0/2555561799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c409c130 con 0x7f59b0004810 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59c37fe640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59b400e7f0 tx=0x7f59b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4003b90 con 0x7f59b0004810 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b4018070 con 0x7f59b0004810 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40050d0 con 0x7f59b0004810 2024-04-02T16:12:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c41c7780 con 0x7f59b0004810 2024-04-02T16:12:14.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41d0710 con 0x7f59b0004810 2024-04-02T16:12:14.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c40685f0 con 0x7f59b0004810 2024-04-02T16:12:14.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.860+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f59b4005270 con 0x7f59b0004810 2024-04-02T16:12:14.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.864+0000 7f59bbfff640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 0x7f59a406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:14.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.864+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5609+0+0 (secure 0 0 0) 0x7f59b4086890 con 0x7f59b0004810 2024-04-02T16:12:14.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.864+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 0x7f59a406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:14.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.864+0000 7f59c2ffd640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 0x7f59a406a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0ae000 tx=0x7f59ac007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:14.868+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f59b4056f40 con 0x7f59b0004810 2024-04-02T16:12:15.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.064+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f59c40b8be0 con 0x7f59b0004810 2024-04-02T16:12:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.692+0000 7f59bbfff640 1 -- 172.21.15.40:0/2555561799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f59b4056840 con 0x7f59b0004810 2024-04-02T16:12:15.692 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 4 and data pool 5 2024-04-02T16:12:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.692+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 msgr2=0x7f59a406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.692+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 0x7f59a406a840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0ae000 tx=0x7f59ac007040 comp rx=0 tx=0).stop 2024-04-02T16:12:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.692+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 msgr2=0x7f59c41d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.692+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59b400e7f0 tx=0x7f59b400ecb0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 shutdown_connections 2024-04-02T16:12:15.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59c41d0f10 0x7f59c409bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f59a4068380 0x7f59a406a840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59c41d2a10 0x7f59c41c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 --2- 172.21.15.40:0/2555561799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b0004810 0x7f59c41d1b20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 >> 172.21.15.40:0/2555561799 conn(0x7f59c40b8fd0 msgr2=0x7f59c406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 shutdown_connections 2024-04-02T16:12:15.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.696+0000 7f59ca002640 1 -- 172.21.15.40:0/2555561799 wait complete. 2024-04-02T16:12:15.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 Processor -- start 2024-04-02T16:12:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 -- start start 2024-04-02T16:12:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f85081500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85080ae830 con 0x7f8508147c50 2024-04-02T16:12:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850806cc80 con 0x7f850806c8a0 2024-04-02T16:12:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f850f13f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f85080b2b50 con 0x7f85081487f0 2024-04-02T16:12:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f8507fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8508147c50 0x7f85081480e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55858/0 (socket says 172.21.15.40:55858) 2024-04-02T16:12:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.856+0000 7f8507fff640 1 -- 172.21.15.40:0/3010971191 learned_addr learned my addr 172.21.15.40:0/3010971191 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f85081500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177358729 0 0) 0x7f85080ae830 con 0x7f8508147c50 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84f0003530 con 0x7f8508147c50 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808475200 0 0) 0x7f84f0003530 con 0x7f8508147c50 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 msgr2=0x7f85081500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f85081500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 >> v1:172.21.15.204:6789/0 conn(0x7f850806c8a0 legacy=0x7f85080af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85080adfa0 con 0x7f8508147c50 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f84fc0016a0 con 0x7f8508147c50 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 >> v1:172.21.15.40:6789/0 conn(0x7f8508147c50 legacy=0x7f85081480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f00039f0 0x7f84f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f84f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f84f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85080b2b50 con 0x7f84f0004d50 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84f000bb90 con 0x7f84f00039f0 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f84f000bd90 con 0x7f85081487f0 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f84f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850cedb640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f00039f0 0x7f84f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f8507fff640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f84f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 -- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 msgr2=0x7f84f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f84f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 -- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f00039f0 msgr2=0x7f84f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f00039f0 0x7f84f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 -- 172.21.15.40:0/3010971191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f0003530 con 0x7f84f0004d50 2024-04-02T16:12:15.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f8507fff640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081487f0 0x7f84f0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f84f00051d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8500009f70 tx=0x7f850001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6821bd875dc1e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85000b2070 con 0x7f84f0004d50 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85000a59d0 con 0x7f84f0004d50 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.860+0000 7f85077fe640 1 -- 172.21.15.40:0/3010971191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500009bf0 con 0x7f84f0004d50 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3010971191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 msgr2=0x7f84f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 --2- 172.21.15.40:0/3010971191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f84f00051d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8500009f70 tx=0x7f850001cc20 comp rx=0 tx=0).stop 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850d6dc640 1 -- 172.21.15.40:0/3010971191 reap_dead start 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3010971191 shutdown_connections 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3010971191 >> 172.21.15.40:0/3010971191 conn(0x7f85080b8fd0 msgr2=0x7f85080b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3010971191 shutdown_connections 2024-04-02T16:12:15.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3010971191 wait complete. 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 Processor -- start 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- start start 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f850809bab0 0x7f85081e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081e1c70 0x7f85081e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85080b54d0 con 0x7f84f0004d50 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85080b08c0 con 0x7f850809bab0 2024-04-02T16:12:15.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f85080b5650 con 0x7f85081e1c70 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59106/0 (socket says 172.21.15.40:59106) 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 -- 172.21.15.40:0/3926145713 learned_addr learned my addr 172.21.15.40:0/3926145713 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850d6dc640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081e1c70 0x7f85081e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f8507fff640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f850809bab0 0x7f85081e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 -- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081e1c70 msgr2=0x7f85081e20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081e1c70 0x7f85081e20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 -- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f850809bab0 msgr2=0x7f85081e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f850809bab0 0x7f85081e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 -- 172.21.15.40:0/3926145713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850809d260 con 0x7f84f0004d50 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850cedb640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e550 tx=0x7f84f800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80052e0 con 0x7f84f0004d50 2024-04-02T16:12:15.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8018070 con 0x7f84f0004d50 2024-04-02T16:12:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850809d4e0 con 0x7f84f0004d50 2024-04-02T16:12:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8003820 con 0x7f84f0004d50 2024-04-02T16:12:15.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850809c350 con 0x7f84f0004d50 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.864+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85080685f0 con 0x7f84f0004d50 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.868+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f84f80039c0 con 0x7f84f0004d50 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.868+0000 7f85057fa640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 0x7f84f406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.868+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f84f80866a0 con 0x7f84f0004d50 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.868+0000 7f8507fff640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 0x7f84f406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:15.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.868+0000 7f8507fff640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 0x7f84f406a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001810 tx=0x7f84fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:15.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:15.872+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f850809d260 con 0x7f84f0004d50 2024-04-02T16:12:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.032+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85080ae610 con 0x7f84f0004d50 2024-04-02T16:12:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f85057fa640 1 -- 172.21.15.40:0/3926145713 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16634 (secure 0 0 0) 0x7f84f8056cf0 con 0x7f84f0004d50 2024-04-02T16:12:16.036 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:16.036 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:15.068111+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 msgr2=0x7f84f406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 0x7f84f406a870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001810 tx=0x7f84fc009040 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 msgr2=0x7f850809cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e550 tx=0x7f84f800ea10 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 shutdown_connections 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85081e1c70 0x7f85081e20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f84f40683b0 0x7f84f406a870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f850809bab0 0x7f85081e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 --2- 172.21.15.40:0/3926145713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84f0004d50 0x7f850809cd20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 >> 172.21.15.40:0/3926145713 conn(0x7f85080b8fd0 msgr2=0x7f85080b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 shutdown_connections 2024-04-02T16:12:16.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.036+0000 7f850f13f640 1 -- 172.21.15.40:0/3926145713 wait complete. 2024-04-02T16:12:16.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:16.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.172+0000 7f8165e6f640 1 Processor -- start 2024-04-02T16:12:16.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.172+0000 7f8165e6f640 1 -- start start 2024-04-02T16:12:16.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f8165e6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b3500 0x7f81600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f8165e6f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81600b8c80 0x7f81600b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f8165e6f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600b6310 con 0x7f81600b4bf0 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f8165e6f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f81600b55d0 con 0x7f81600b8c80 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f8165e6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f816014b560 con 0x7f81600b3500 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81600b8c80 0x7f81600b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81600b4bf0 0x7f816014ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55868/0 (socket says 172.21.15.40:55868) 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815f7fe640 1 -- 172.21.15.40:0/1349142647 learned_addr learned my addr 172.21.15.40:0/1349142647 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:16.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815effd640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b3500 0x7f81600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 -- 172.21.15.40:0/1349142647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b3500 msgr2=0x7f81600b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b3500 0x7f81600b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 -- 172.21.15.40:0/1349142647 >> v1:172.21.15.40:6789/0 conn(0x7f81600b4bf0 legacy=0x7f816014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 -- 172.21.15.40:0/1349142647 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81600b3fa0 con 0x7f81600b8c80 2024-04-02T16:12:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815e7fc640 1 -- 172.21.15.40:0/1349142647 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619581839 0 0) 0x7f81600b6310 con 0x7f81600b4bf0 2024-04-02T16:12:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815ffff640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b8c80 0x7f81600b9130 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8158009d90 tx=0x7f815801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35aaf6c1ffdadf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815e7fc640 1 -- 172.21.15.40:0/1349142647 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580b2070 con 0x7f81600b8c80 2024-04-02T16:12:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815e7fc640 1 -- 172.21.15.40:0/1349142647 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81580a5710 con 0x7f81600b8c80 2024-04-02T16:12:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.176+0000 7f815e7fc640 1 -- 172.21.15.40:0/1349142647 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81580a5ae0 con 0x7f81600b8c80 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/1349142647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b8c80 msgr2=0x7f81600b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b8c80 0x7f81600b9130 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8158009d90 tx=0x7f815801c850 comp rx=0 tx=0).stop 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/1349142647 shutdown_connections 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b8c80 0x7f81600b9130 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- 172.21.15.40:0/1349142647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b3500 0x7f81600b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/1349142647 >> 172.21.15.40:0/1349142647 conn(0x7f81600734b0 msgr2=0x7f81600bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/1349142647 shutdown_connections 2024-04-02T16:12:16.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/1349142647 wait complete. 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 Processor -- start 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- start start 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b4bf0 0x7f8160143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 0x7f8160145710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8160154d90 con 0x7f81600b3500 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8160154c10 con 0x7f81600b8c80 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8160154f10 con 0x7f81600b4bf0 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59136/0 (socket says 172.21.15.40:59136) 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 -- 172.21.15.40:0/262478929 learned_addr learned my addr 172.21.15.40:0/262478929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815ffff640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 0x7f8160145710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815effd640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b4bf0 0x7f8160143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 -- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b4bf0 msgr2=0x7f8160143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b4bf0 0x7f8160143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 -- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 msgr2=0x7f8160145710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 0x7f8160145710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 -- 172.21.15.40:0/262478929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8160145c50 con 0x7f81600b3500 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815ffff640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 0x7f8160145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f815f7fe640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81540011f0 tx=0x7f815400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154018070 con 0x7f81600b3500 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81540074e0 con 0x7f81600b3500 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816014c6e0 con 0x7f81600b3500 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154004a10 con 0x7f81600b3500 2024-04-02T16:12:16.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81601485d0 con 0x7f81600b3500 2024-04-02T16:12:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.180+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8160144820 con 0x7f81600b3500 2024-04-02T16:12:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.184+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f815401b070 con 0x7f81600b3500 2024-04-02T16:12:16.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.184+0000 7f8147fff640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 0x7f814c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.184+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f8154085ce0 con 0x7f81600b3500 2024-04-02T16:12:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.184+0000 7f815effd640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 0x7f814c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.184+0000 7f815effd640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 0x7f814c06a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81601e4170 tx=0x7f8150008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.188+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f8154056330 con 0x7f81600b3500 2024-04-02T16:12:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.348+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81600685f0 con 0x7f81600b3500 2024-04-02T16:12:16.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.352+0000 7f8147fff640 1 -- 172.21.15.40:0/262478929 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16634 (secure 0 0 0) 0x7f8154055c30 con 0x7f81600b3500 2024-04-02T16:12:16.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":28,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:15.068111+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.352+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 msgr2=0x7f814c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.352+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 0x7f814c06a840 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f81601e4170 tx=0x7f8150008040 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.352+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 msgr2=0x7f816014c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.352+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81540011f0 tx=0x7f815400c9b0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 shutdown_connections 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81600b4bf0 0x7f8160143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f814c068380 0x7f814c06a840 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81600b8c80 0x7f8160145710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 --2- 172.21.15.40:0/262478929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81600b3500 0x7f816014c080 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 >> 172.21.15.40:0/262478929 conn(0x7f81600734b0 msgr2=0x7f8160141570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 shutdown_connections 2024-04-02T16:12:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.356+0000 7f8165e6f640 1 -- 172.21.15.40:0/262478929 wait complete. 2024-04-02T16:12:16.373 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T16:12:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 Processor -- start 2024-04-02T16:12:16.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 -- start start 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f43300b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300b19e0 con 0x7f43300b23d0 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300b05c0 con 0x7f4330148e50 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4337504640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f43300b08a0 con 0x7f4330073230 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f43352a0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43300b23d0 0x7f433009daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55874/0 (socket says 172.21.15.40:55874) 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f43352a0640 1 -- 172.21.15.40:0/1623957423 learned_addr learned my addr 172.21.15.40:0/1623957423 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4334a9f640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f43300b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1710991533 0 0) 0x7f43300b19e0 con 0x7f43300b23d0 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f431c003530 con 0x7f43300b23d0 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1928783198 0 0) 0x7f431c003530 con 0x7f43300b23d0 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 msgr2=0x7f43300b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f43300b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 >> v1:172.21.15.204:6789/0 conn(0x7f4330148e50 legacy=0x7f43301492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300b0b80 con 0x7f43300b23d0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f432c001a00 con 0x7f43300b23d0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 >> v1:172.21.15.40:6789/0 conn(0x7f43300b23d0 legacy=0x7f433009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431c0039f0 0x7f431c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f431c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c004d50 0x7f431c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43300b08a0 con 0x7f431c004d50 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f431c00bb30 con 0x7f431c0039f0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f431c00bd30 con 0x7f4330073230 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c004d50 0x7f431c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f43352a0640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431c0039f0 0x7f431c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4334a9f640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f431c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 -- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 msgr2=0x7f431c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4330073230 0x7f431c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 -- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431c0039f0 msgr2=0x7f431c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f431c0039f0 0x7f431c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.496+0000 7f4335aa1640 1 -- 172.21.15.40:0/1623957423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c003530 con 0x7f431c004d50 2024-04-02T16:12:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c004d50 0x7f431c0051d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4328001780 tx=0x7f432801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fd93d87ff74adf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280013a0 con 0x7f431c004d50 2024-04-02T16:12:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280052f0 con 0x7f431c004d50 2024-04-02T16:12:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4323fff640 1 -- 172.21.15.40:0/1623957423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328003840 con 0x7f431c004d50 2024-04-02T16:12:16.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- 172.21.15.40:0/1623957423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c004d50 msgr2=0x7f431c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 --2- 172.21.15.40:0/1623957423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c004d50 0x7f431c0051d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4328001780 tx=0x7f432801c420 comp rx=0 tx=0).stop 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4335aa1640 1 -- 172.21.15.40:0/1623957423 reap_dead start 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- 172.21.15.40:0/1623957423 shutdown_connections 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- 172.21.15.40:0/1623957423 >> 172.21.15.40:0/1623957423 conn(0x7f43300b8fd0 msgr2=0x7f43300b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- 172.21.15.40:0/1623957423 shutdown_connections 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- 172.21.15.40:0/1623957423 wait complete. 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 Processor -- start 2024-04-02T16:12:16.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.500+0000 7f4337504640 1 -- start start 2024-04-02T16:12:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f431c004d50 0x7f43301d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 0x7f43301cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43300b3800 con 0x7f43301cf510 2024-04-02T16:12:16.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43300b27b0 con 0x7f433009d2f0 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f43301cf990 con 0x7f431c004d50 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59154/0 (socket says 172.21.15.40:59154) 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 -- 172.21.15.40:0/1128013717 learned_addr learned my addr 172.21.15.40:0/1128013717 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4334a9f640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 0x7f43301cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f43352a0640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f431c004d50 0x7f43301d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 -- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f431c004d50 msgr2=0x7f43301d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f431c004d50 0x7f43301d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 -- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 msgr2=0x7f43301cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 0x7f43301cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 -- 172.21.15.40:0/1128013717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43301d06a0 con 0x7f43301cf510 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4334a9f640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 0x7f43301cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4335aa1640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4328001780 tx=0x7f431c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280ba020 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280054a0 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433009d860 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280013a0 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43301d02a0 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43300685f0 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f43280bd070 con 0x7f43301cf510 2024-04-02T16:12:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 0x7f431406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f4328120090 con 0x7f43301cf510 2024-04-02T16:12:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.504+0000 7f43352a0640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 0x7f431406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.508+0000 7f43352a0640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 0x7f431406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f432c002240 tx=0x7f432c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.508+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f4328016130 con 0x7f43301cf510 2024-04-02T16:12:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.712+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f43301ce910 con 0x7f43301cf510 2024-04-02T16:12:16.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.724+0000 7f4321ffb640 1 -- 172.21.15.40:0/1128013717 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7f43280f0650 con 0x7f43301cf510 2024-04-02T16:12:16.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.724+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 msgr2=0x7f431406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.724+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 0x7f431406a840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f432c002240 tx=0x7f432c009040 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.724+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 msgr2=0x7f43301ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.724+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4328001780 tx=0x7f431c005840 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 shutdown_connections 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4314068380 0x7f431406a840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43301cf510 0x7f43301ded90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f433009d2f0 0x7f43301cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 --2- 172.21.15.40:0/1128013717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f431c004d50 0x7f43301d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 >> 172.21.15.40:0/1128013717 conn(0x7f43300b8fd0 msgr2=0x7f43300b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 shutdown_connections 2024-04-02T16:12:16.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.728+0000 7f4337504640 1 -- 172.21.15.40:0/1128013717 wait complete. 2024-04-02T16:12:16.744 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-02T16:12:16.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-02T16:12:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.860+0000 7f29ea03f640 1 Processor -- start 2024-04-02T16:12:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.860+0000 7f29ea03f640 1 -- start start 2024-04-02T16:12:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29ea03f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29e40b4bf0 0x7f29e40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29ea03f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40b6310 con 0x7f29e40b3500 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29ea03f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40b55d0 con 0x7f29e40b8c80 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29ea03f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f29e414b560 con 0x7f29e40b4bf0 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29daffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f29e40b3500 0x7f29e414ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55882/0 (socket says 172.21.15.40:55882) 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29daffd640 1 -- 172.21.15.40:0/2575998799 learned_addr learned my addr 172.21.15.40:0/2575998799 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e37fe640 1 --2- 172.21.15.40:0/2575998799 >> v2:172.21.15.204:3300/0 conn(0x7f29e40b4bf0 0x7f29e40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713409102 0 0) 0x7f29e40b6310 con 0x7f29e40b3500 2024-04-02T16:12:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29d0003530 con 0x7f29e40b3500 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1441766749 0 0) 0x7f29d0003530 con 0x7f29e40b3500 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 >> v2:172.21.15.204:3300/0 conn(0x7f29e40b4bf0 msgr2=0x7f29e40b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 --2- 172.21.15.40:0/2575998799 >> v2:172.21.15.204:3300/0 conn(0x7f29e40b4bf0 0x7f29e40b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 >> v1:172.21.15.204:6789/0 conn(0x7f29e40b8c80 legacy=0x7f29e41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40b3fa0 con 0x7f29e40b3500 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f29cc001680 con 0x7f29e40b3500 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 >> v1:172.21.15.40:6789/0 conn(0x7f29e40b3500 legacy=0x7f29e414ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00039c0 0x7f29d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d0004810 0x7f29d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29e414b560 con 0x7f29d00052f0 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29d000c1c0 con 0x7f29d0004810 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29d000c3c0 con 0x7f29d00039c0 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29daffd640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d0004810 0x7f29d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e37fe640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00039c0 0x7f29d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 -- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00039c0 msgr2=0x7f29d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00039c0 0x7f29d0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 -- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d0004810 msgr2=0x7f29d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d0004810 0x7f29d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.864+0000 7f29e3fff640 1 -- 172.21.15.40:0/2575998799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d0003530 con 0x7f29d00052f0 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e3fff640 1 -- 172.21.15.40:0/2575998799 reap_dead start 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e3fff640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29d000b9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f29dc001e40 tx=0x7f29dc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cd30f9cd638210 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc001a90 con 0x7f29d00052f0 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc0a7070 con 0x7f29d00052f0 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e2ffd640 1 -- 172.21.15.40:0/2575998799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc003da0 con 0x7f29d00052f0 2024-04-02T16:12:16.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 msgr2=0x7f29d000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29d000b9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f29dc001e40 tx=0x7f29dc01c760 comp rx=0 tx=0).stop 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/2575998799 shutdown_connections 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 --2- 172.21.15.40:0/2575998799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29d000b9d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/2575998799 >> 172.21.15.40:0/2575998799 conn(0x7f29e40734b0 msgr2=0x7f29e40bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/2575998799 shutdown_connections 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/2575998799 wait complete. 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 Processor -- start 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- start start 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29e41e0400 0x7f29e41e0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e41e3970 0x7f29e41e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40b5360 con 0x7f29d00052f0 2024-04-02T16:12:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40b51e0 con 0x7f29e41e3970 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29e4144e60 con 0x7f29e41e0400 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59172/0 (socket says 172.21.15.40:59172) 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 -- 172.21.15.40:0/4254601038 learned_addr learned my addr 172.21.15.40:0/4254601038 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e3fff640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e41e3970 0x7f29e41e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29daffd640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29e41e0400 0x7f29e41e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 -- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29e41e0400 msgr2=0x7f29e41e0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29e41e0400 0x7f29e41e0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 -- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e41e3970 msgr2=0x7f29e41e3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e41e3970 0x7f29e41e3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 -- 172.21.15.40:0/4254601038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e413e3c0 con 0x7f29d00052f0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e37fe640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29d40108c0 tx=0x7f29d4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400f040 con 0x7f29d00052f0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d400eb60 con 0x7f29d00052f0 2024-04-02T16:12:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d401c400 con 0x7f29d00052f0 2024-04-02T16:12:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e41445e0 con 0x7f29d00052f0 2024-04-02T16:12:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.868+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e4148100 con 0x7f29d00052f0 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40685f0 con 0x7f29d00052f0 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f29d401c5a0 con 0x7f29d00052f0 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29e0ff9640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 0x7f29c406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f29d4086fe0 con 0x7f29d00052f0 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29daffd640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 0x7f29c406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.872+0000 7f29daffd640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 0x7f29c406a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0010f0 tx=0x7f29cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:16.876+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f29d4018030 con 0x7f29d00052f0 2024-04-02T16:12:17.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.068+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f29e40baf30 con 0x7f29d00052f0 2024-04-02T16:12:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.732+0000 7f29e0ff9640 1 -- 172.21.15.40:0/4254601038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f29d4057630 con 0x7f29d00052f0 2024-04-02T16:12:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 msgr2=0x7f29c406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 0x7f29c406a870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0010f0 tx=0x7f29cc009040 comp rx=0 tx=0).stop 2024-04-02T16:12:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 msgr2=0x7f29e4147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f29d40108c0 tx=0x7f29d4010d80 comp rx=0 tx=0).stop 2024-04-02T16:12:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 shutdown_connections 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29e41e0400 0x7f29e41e0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f29c40683b0 0x7f29c406a870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29e41e3970 0x7f29e41e3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 --2- 172.21.15.40:0/4254601038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d00052f0 0x7f29e4147610 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 >> 172.21.15.40:0/4254601038 conn(0x7f29e40734b0 msgr2=0x7f29e413c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 shutdown_connections 2024-04-02T16:12:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.736+0000 7f29ea03f640 1 -- 172.21.15.40:0/4254601038 wait complete. 2024-04-02T16:12:17.756 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-02T16:12:17.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:17.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.876+0000 7f3c632fe640 1 Processor -- start 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 -- start start 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3c5c0b23d0 0x7f3c5c0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c5c073230 0x7f3c5c09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5c0b19e0 con 0x7f3c5c073230 2024-04-02T16:12:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5c0b05c0 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c632fe640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5c0b08a0 con 0x7f3c5c09dfe0 2024-04-02T16:12:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c5c073230 0x7f3c5c09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c5c073230 0x7f3c5c09daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59178/0 (socket says 172.21.15.40:59178) 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 -- 172.21.15.40:0/3877914213 learned_addr learned my addr 172.21.15.40:0/3877914213 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c6109a640 1 --2- 172.21.15.40:0/3877914213 >> v2:172.21.15.204:3300/0 conn(0x7f3c5c0b23d0 0x7f3c5c0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c6189b640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 -- 172.21.15.40:0/3877914213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 msgr2=0x7f3c5c148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 -- 172.21.15.40:0/3877914213 >> v2:172.21.15.204:3300/0 conn(0x7f3c5c0b23d0 msgr2=0x7f3c5c0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 --2- 172.21.15.40:0/3877914213 >> v2:172.21.15.204:3300/0 conn(0x7f3c5c0b23d0 0x7f3c5c0b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 -- 172.21.15.40:0/3877914213 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c0b0b80 con 0x7f3c5c073230 2024-04-02T16:12:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c6189b640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.880+0000 7f3c60899640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c073230 0x7f3c5c09daa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0b00c0 tx=0x7f3c4c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2734b01f4df630 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c53fff640 1 -- 172.21.15.40:0/3877914213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0b0070 con 0x7f3c5c073230 2024-04-02T16:12:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c53fff640 1 -- 172.21.15.40:0/3877914213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0ab600 con 0x7f3c5c073230 2024-04-02T16:12:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c53fff640 1 -- 172.21.15.40:0/3877914213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0ab9d0 con 0x7f3c5c073230 2024-04-02T16:12:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/3877914213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c073230 msgr2=0x7f3c5c09daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c073230 0x7f3c5c09daa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0b00c0 tx=0x7f3c4c01c490 comp rx=0 tx=0).stop 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/3877914213 shutdown_connections 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- 172.21.15.40:0/3877914213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c073230 0x7f3c5c09daa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- 172.21.15.40:0/3877914213 >> v2:172.21.15.204:3300/0 conn(0x7f3c5c0b23d0 0x7f3c5c0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/3877914213 >> 172.21.15.40:0/3877914213 conn(0x7f3c5c0b8fd0 msgr2=0x7f3c5c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/3877914213 shutdown_connections 2024-04-02T16:12:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/3877914213 wait complete. 2024-04-02T16:12:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 Processor -- start 2024-04-02T16:12:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- start start 2024-04-02T16:12:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c5c073230 0x7f3c5c13a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c1df1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1460f0 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c145f70 con 0x7f3c5c073230 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5c146270 con 0x7f3c5c09dfe0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59192/0 (socket says 172.21.15.40:59192) 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 -- 172.21.15.40:0/2555264604 learned_addr learned my addr 172.21.15.40:0/2555264604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6109a640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c5c073230 0x7f3c5c13a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c60899640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c1df1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 -- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 msgr2=0x7f3c5c1df1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c1df1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 -- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c5c073230 msgr2=0x7f3c5c13a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c5c073230 0x7f3c5c13a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 -- 172.21.15.40:0/2555264604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c06fa40 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c6189b640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009fd0 tx=0x7f3c5400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5401a070 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c54003bc0 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c13ba60 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004030 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.884+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c13ab00 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5c0685f0 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3c5401d070 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c51ffb640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 0x7f3c4006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3c54086c70 con 0x7f3c5c0b23d0 2024-04-02T16:12:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c6109a640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 0x7f3c4006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.888+0000 7f3c6109a640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 0x7f3c4006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c580099b0 tx=0x7f3c58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:17.892+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3c54010070 con 0x7f3c5c0b23d0 2024-04-02T16:12:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.084+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c5c0b30b0 con 0x7f3c5c0b23d0 2024-04-02T16:12:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.084+0000 7f3c51ffb640 1 -- 172.21.15.40:0/2555264604 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7f3c540572c0 con 0x7f3c5c0b23d0 2024-04-02T16:12:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:18.086 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:18.086 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 msgr2=0x7f3c4006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 0x7f3c4006a840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c580099b0 tx=0x7f3c58008040 comp rx=0 tx=0).stop 2024-04-02T16:12:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 msgr2=0x7f3c5c1df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009fd0 tx=0x7f3c5400d560 comp rx=0 tx=0).stop 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 shutdown_connections 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c5c09dfe0 0x7f3c5c1df1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3c40068380 0x7f3c4006a840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c5c0b23d0 0x7f3c5c1df720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 --2- 172.21.15.40:0/2555264604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c5c073230 0x7f3c5c13a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 >> 172.21.15.40:0/2555264604 conn(0x7f3c5c0b8fd0 msgr2=0x7f3c5c06cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 shutdown_connections 2024-04-02T16:12:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.088+0000 7f3c632fe640 1 -- 172.21.15.40:0/2555264604 wait complete. 2024-04-02T16:12:18.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 Processor -- start 2024-04-02T16:12:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 -- start start 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95e40b4f00 0x7f95e40bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95e414fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e40b52e0 con 0x7f95e40b6170 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f95e40b6f00 con 0x7f95e40b4f00 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95ea06a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95e40b71a0 con 0x7f95e4149590 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95e2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f95e40b6170 0x7f95e4148e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55892/0 (socket says 172.21.15.40:55892) 2024-04-02T16:12:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.212+0000 7f95e2ffd640 1 -- 172.21.15.40:0/4133638875 learned_addr learned my addr 172.21.15.40:0/4133638875 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4133638875 >> v2:172.21.15.204:3300/0 conn(0x7f95e40b4f00 0x7f95e40bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4280334063 0 0) 0x7f95d8001120 con 0x7f95e40b6170 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95e414fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95cc003530 con 0x7f95e40b6170 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4161986611 0 0) 0x7f95cc003530 con 0x7f95e40b6170 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 msgr2=0x7f95e414fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95e414fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 >> v2:172.21.15.204:3300/0 conn(0x7f95e40b4f00 msgr2=0x7f95e40bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 --2- 172.21.15.40:0/4133638875 >> v2:172.21.15.204:3300/0 conn(0x7f95e40b4f00 0x7f95e40bb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e40b7480 con 0x7f95e40b6170 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f95d8001850 con 0x7f95e40b6170 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 >> v1:172.21.15.40:6789/0 conn(0x7f95e40b6170 legacy=0x7f95e4148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039c0 0x7f95cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004d50 0x7f95cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40b6f00 con 0x7f95cc004d50 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40b71a0 con 0x7f95cc0039c0 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc00ba50 con 0x7f95e4149590 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004d50 0x7f95cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e2ffd640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039c0 0x7f95cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 -- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 msgr2=0x7f95cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 -- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039c0 msgr2=0x7f95cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039c0 0x7f95cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 -- 172.21.15.40:0/4133638875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc003530 con 0x7f95cc004d50 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95cc0039c0 0x7f95cc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:18.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e2ffd640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e4149590 0x7f95cc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:18.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004d50 0x7f95cc0051d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009fc0 tx=0x7f95dc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5cb961cfad18e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0b2070 con 0x7f95cc004d50 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95dc0a5a70 con 0x7f95cc004d50 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e27fc640 1 -- 172.21.15.40:0/4133638875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc009c10 con 0x7f95cc004d50 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 -- 172.21.15.40:0/4133638875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004d50 msgr2=0x7f95cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4133638875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95cc004d50 0x7f95cc0051d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009fc0 tx=0x7f95dc01cc80 comp rx=0 tx=0).stop 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95e3fff640 1 -- 172.21.15.40:0/4133638875 reap_dead start 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 -- 172.21.15.40:0/4133638875 shutdown_connections 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 -- 172.21.15.40:0/4133638875 >> 172.21.15.40:0/4133638875 conn(0x7f95e402c5d0 msgr2=0x7f95e402c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 -- 172.21.15.40:0/4133638875 shutdown_connections 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.216+0000 7f95ea06a640 1 -- 172.21.15.40:0/4133638875 wait complete. 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 Processor -- start 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- start start 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e41ddf00 0x7f95e4133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e41344f0 0x7f95e41e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4072b90 con 0x7f95e41392a0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40b59e0 con 0x7f95e41344f0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95e4150650 con 0x7f95e41ddf00 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59206/0 (socket says 172.21.15.40:59206) 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 -- 172.21.15.40:0/4208785323 learned_addr learned my addr 172.21.15.40:0/4208785323 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e2ffd640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e41ddf00 0x7f95e4133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e3fff640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e41344f0 0x7f95e41e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 -- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e41ddf00 msgr2=0x7f95e4133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e41ddf00 0x7f95e4133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 -- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e41344f0 msgr2=0x7f95e41e0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e41344f0 0x7f95e41e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 -- 172.21.15.40:0/4208785323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e4134970 con 0x7f95e41392a0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95e37fe640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f95d4010950 tx=0x7f95d4010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400f040 con 0x7f95e41392a0 2024-04-02T16:12:18.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d400ec20 con 0x7f95e41392a0 2024-04-02T16:12:18.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e4139680 con 0x7f95e41392a0 2024-04-02T16:12:18.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4004020 con 0x7f95e41392a0 2024-04-02T16:12:18.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e413bbd0 con 0x7f95e41392a0 2024-04-02T16:12:18.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e4068d00 con 0x7f95e41392a0 2024-04-02T16:12:18.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.220+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f95d40041c0 con 0x7f95e41392a0 2024-04-02T16:12:18.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.224+0000 7f95cbfff640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.224+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f95d4086a00 con 0x7f95e41392a0 2024-04-02T16:12:18.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.224+0000 7f95e2ffd640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.224+0000 7f95e2ffd640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 0x7f95d006a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000fc0 tx=0x7f95d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.228+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f95d4018030 con 0x7f95e41392a0 2024-04-02T16:12:18.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.380+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95e414dc10 con 0x7f95e41392a0 2024-04-02T16:12:18.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.384+0000 7f95cbfff640 1 -- 172.21.15.40:0/4208785323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16621 (secure 0 0 0) 0x7f95d40044b0 con 0x7f95e41392a0 2024-04-02T16:12:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:18.387 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:16.692754+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:18.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 msgr2=0x7f95d006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 0x7f95d006a870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000fc0 tx=0x7f95d8009040 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 msgr2=0x7f95e41dd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f95d4010950 tx=0x7f95d4010e10 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 shutdown_connections 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95e41ddf00 0x7f95e4133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f95d00683b0 0x7f95d006a870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95e41344f0 0x7f95e41e0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 --2- 172.21.15.40:0/4208785323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95e41392a0 0x7f95e41dd9c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 >> 172.21.15.40:0/4208785323 conn(0x7f95e402c5d0 msgr2=0x7f95e40672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 shutdown_connections 2024-04-02T16:12:18.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.388+0000 7f95ea06a640 1 -- 172.21.15.40:0/4208785323 wait complete. 2024-04-02T16:12:18.404 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T16:12:18.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 Processor -- start 2024-04-02T16:12:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 -- start start 2024-04-02T16:12:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fefa00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa00b19e0 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa00b05c0 con 0x7fefa0073230 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fefa5586640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefa00b08a0 con 0x7fefa00b23d0 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefa0148e50 0x7fefa01492c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55898/0 (socket says 172.21.15.40:55898) 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9f7fe640 1 -- 172.21.15.40:0/515557360 learned_addr learned my addr 172.21.15.40:0/515557360 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9effd640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fefa00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095113606 0 0) 0x7fefa00b19e0 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef88003530 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3990439837 0 0) 0x7fef88003530 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 msgr2=0x7fefa00b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fefa00b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 >> v1:172.21.15.204:6789/0 conn(0x7fefa0073230 legacy=0x7fefa009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa00b0b80 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fef98001a00 con 0x7fefa0148e50 2024-04-02T16:12:18.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 >> v1:172.21.15.40:6789/0 conn(0x7fefa0148e50 legacy=0x7fefa01492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef880039f0 0x7fef88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880043a0 0x7fef88004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fef88005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b08a0 con 0x7fef880043a0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fef8800ba80 con 0x7fef880039f0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fef8800bc80 con 0x7fefa00b23d0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880043a0 0x7fef88004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9effd640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef880039f0 0x7fef88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9f7fe640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fef88005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 -- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 msgr2=0x7fef88005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fef88005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 -- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef880039f0 msgr2=0x7fef88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef880039f0 0x7fef88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9e7fc640 1 -- 172.21.15.40:0/515557360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef88003530 con 0x7fef880043a0 2024-04-02T16:12:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.524+0000 7fef9f7fe640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fefa00b23d0 0x7fef88005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880043a0 0x7fef88004870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fefa0073650 tx=0x7fef9401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54c7652a8a9c637 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef940b5070 con 0x7fef880043a0 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef940a88f0 con 0x7fef880043a0 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9dffb640 1 -- 172.21.15.40:0/515557360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94001510 con 0x7fef880043a0 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- 172.21.15.40:0/515557360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880043a0 msgr2=0x7fef88004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 --2- 172.21.15.40:0/515557360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef880043a0 0x7fef88004870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fefa0073650 tx=0x7fef9401ecb0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9f7fe640 1 -- 172.21.15.40:0/515557360 reap_dead start 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- 172.21.15.40:0/515557360 shutdown_connections 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- 172.21.15.40:0/515557360 >> 172.21.15.40:0/515557360 conn(0x7fefa00b8fd0 msgr2=0x7fefa00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- 172.21.15.40:0/515557360 shutdown_connections 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- 172.21.15.40:0/515557360 wait complete. 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 Processor -- start 2024-04-02T16:12:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- start start 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef880043a0 0x7fefa009b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefa0145bc0 0x7fefa01d5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b3800 con 0x7fefa009d830 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fefa00b27b0 con 0x7fefa0145bc0 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fefa5586640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fefa009cf50 con 0x7fef880043a0 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59230/0 (socket says 172.21.15.40:59230) 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.528+0000 7fef9e7fc640 1 -- 172.21.15.40:0/3343132781 learned_addr learned my addr 172.21.15.40:0/3343132781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9effd640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef880043a0 0x7fefa009b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9f7fe640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefa0145bc0 0x7fefa01d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 -- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef880043a0 msgr2=0x7fefa009b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef880043a0 0x7fefa009b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 -- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefa0145bc0 msgr2=0x7fefa01d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefa0145bc0 0x7fefa01d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 -- 172.21.15.40:0/3343132781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa009bc40 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9e7fc640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef9401eae0 tx=0x7fef88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef940b5040 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef940c0070 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa009cc70 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef940052e0 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa0145060 con 0x7fefa009d830 2024-04-02T16:12:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa00685f0 con 0x7fefa009d830 2024-04-02T16:12:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fef94003600 con 0x7fefa009d830 2024-04-02T16:12:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 0x7fef8c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fef941241f0 con 0x7fefa009d830 2024-04-02T16:12:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9effd640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 0x7fef8c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.532+0000 7fef9effd640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 0x7fef8c06a870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fefa009b0e0 tx=0x7fef90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.536+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fef940189c0 con 0x7fefa009d830 2024-04-02T16:12:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.700+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefa009c300 con 0x7fef8c0683b0 2024-04-02T16:12:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.716+0000 7fef877fe640 1 -- 172.21.15.40:0/3343132781 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fefa009c300 con 0x7fef8c0683b0 2024-04-02T16:12:18.722 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 msgr2=0x7fef8c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 0x7fef8c06a870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fefa009b0e0 tx=0x7fef90008040 comp rx=0 tx=0).stop 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 msgr2=0x7fefa01db270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef9401eae0 tx=0x7fef88005840 comp rx=0 tx=0).stop 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 shutdown_connections 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fef8c0683b0 0x7fef8c06a870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefa0145bc0 0x7fefa01d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefa009d830 0x7fefa01db270 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 --2- 172.21.15.40:0/3343132781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef880043a0 0x7fefa009b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 >> 172.21.15.40:0/3343132781 conn(0x7fefa00b8fd0 msgr2=0x7fefa006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 shutdown_connections 2024-04-02T16:12:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.724+0000 7fefa5586640 1 -- 172.21.15.40:0/3343132781 wait complete. 2024-04-02T16:12:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-04-02T16:12:17.863375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219596,"kb_used_data":3436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755124,"statfs":{"total":773094113280,"available":772869246976,"internally_reserved":0,"allocated":3518464,"data_stored":2036897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.181773"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.713236+0000","last_change":"2024-04-02T16:12:14.713236+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.713236+0000","last_undegraded":"2024-04-02T16:12:14.713236+0000","last_fullsized":"2024-04-02T16:12:14.713236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703207+0000","last_change":"2024-04-02T16:12:14.703207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703207+0000","last_undegraded":"2024-04-02T16:12:14.703207+0000","last_fullsized":"2024-04-02T16:12:14.703207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710853+0000","last_change":"2024-04-02T16:12:14.710853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710853+0000","last_undegraded":"2024-04-02T16:12:14.710853+0000","last_fullsized":"2024-04-02T16:12:14.710853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712715+0000","last_change":"2024-04-02T16:12:14.712715+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712715+0000","last_undegraded":"2024-04-02T16:12:14.712715+0000","last_fullsized":"2024-04-02T16:12:14.712715+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699530+0000","last_change":"2024-04-02T16:12:14.699530+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699530+0000","last_undegraded":"2024-04-02T16:12:14.699530+0000","last_fullsized":"2024-04-02T16:12:14.699530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698314+0000","last_change":"2024-04-02T16:12:14.698314+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698314+0000","last_undegraded":"2024-04-02T16:12:14.698314+0000","last_fullsized":"2024-04-02T16:12:14.698314+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704155+0000","last_change":"2024-04-02T16:12:14.704155+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704155+0000","last_undegraded":"2024-04-02T16:12:14.704155+0000","last_fullsized":"2024-04-02T16:12:14.704155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.719220+0000","last_change":"2024-04-02T16:12:14.719220+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.719220+0000","last_undegraded":"2024-04-02T16:12:14.719220+0000","last_fullsized":"2024-04-02T16:12:14.719220+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699751+0000","last_change":"2024-04-02T16:12:14.699751+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699751+0000","last_undegraded":"2024-04-02T16:12:14.699751+0000","last_fullsized":"2024-04-02T16:12:14.699751+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705979+0000","last_change":"2024-04-02T16:12:14.705979+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705979+0000","last_undegraded":"2024-04-02T16:12:14.705979+0000","last_fullsized":"2024-04-02T16:12:14.705979+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708302+0000","last_change":"2024-04-02T16:12:14.708302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708302+0000","last_undegraded":"2024-04-02T16:12:14.708302+0000","last_fullsized":"2024-04-02T16:12:14.708302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707900+0000","last_change":"2024-04-02T16:12:14.707900+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707900+0000","last_undegraded":"2024-04-02T16:12:14.707900+0000","last_fullsized":"2024-04-02T16:12:14.707900+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698803+0000","last_change":"2024-04-02T16:12:14.698803+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698803+0000","last_undegraded":"2024-04-02T16:12:14.698803+0000","last_fullsized":"2024-04-02T16:12:14.698803+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697784+0000","last_change":"2024-04-02T16:12:14.697784+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697784+0000","last_undegraded":"2024-04-02T16:12:14.697784+0000","last_fullsized":"2024-04-02T16:12:14.697784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710747+0000","last_change":"2024-04-02T16:12:14.710747+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710747+0000","last_undegraded":"2024-04-02T16:12:14.710747+0000","last_fullsized":"2024-04-02T16:12:14.710747+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704126+0000","last_change":"2024-04-02T16:12:14.704126+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704126+0000","last_undegraded":"2024-04-02T16:12:14.704126+0000","last_fullsized":"2024-04-02T16:12:14.704126+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706721+0000","last_change":"2024-04-02T16:12:14.706721+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706721+0000","last_undegraded":"2024-04-02T16:12:14.706721+0000","last_fullsized":"2024-04-02T16:12:14.706721+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702833+0000","last_change":"2024-04-02T16:12:14.702833+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702833+0000","last_undegraded":"2024-04-02T16:12:14.702833+0000","last_fullsized":"2024-04-02T16:12:14.702833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702324+0000","last_change":"2024-04-02T16:12:14.702324+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702324+0000","last_undegraded":"2024-04-02T16:12:14.702324+0000","last_fullsized":"2024-04-02T16:12:14.702324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697853+0000","last_change":"2024-04-02T16:12:14.697853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697853+0000","last_undegraded":"2024-04-02T16:12:14.697853+0000","last_fullsized":"2024-04-02T16:12:14.697853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703930+0000","last_change":"2024-04-02T16:12:14.703930+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703930+0000","last_undegraded":"2024-04-02T16:12:14.703930+0000","last_fullsized":"2024-04-02T16:12:14.703930+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700364+0000","last_change":"2024-04-02T16:12:14.700364+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700364+0000","last_undegraded":"2024-04-02T16:12:14.700364+0000","last_fullsized":"2024-04-02T16:12:14.700364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699911+0000","last_change":"2024-04-02T16:12:14.699911+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699911+0000","last_undegraded":"2024-04-02T16:12:14.699911+0000","last_fullsized":"2024-04-02T16:12:14.699911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705612+0000","last_change":"2024-04-02T16:12:14.705612+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705612+0000","last_undegraded":"2024-04-02T16:12:14.705612+0000","last_fullsized":"2024-04-02T16:12:14.705612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700630+0000","last_change":"2024-04-02T16:12:14.700630+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700630+0000","last_undegraded":"2024-04-02T16:12:14.700630+0000","last_fullsized":"2024-04-02T16:12:14.700630+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698892+0000","last_change":"2024-04-02T16:12:14.698892+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698892+0000","last_undegraded":"2024-04-02T16:12:14.698892+0000","last_fullsized":"2024-04-02T16:12:14.698892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698461+0000","last_change":"2024-04-02T16:12:14.698461+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698461+0000","last_undegraded":"2024-04-02T16:12:14.698461+0000","last_fullsized":"2024-04-02T16:12:14.698461+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706731+0000","last_change":"2024-04-02T16:12:14.706731+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706731+0000","last_undegraded":"2024-04-02T16:12:14.706731+0000","last_fullsized":"2024-04-02T16:12:14.706731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702801+0000","last_change":"2024-04-02T16:12:14.702801+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702801+0000","last_undegraded":"2024-04-02T16:12:14.702801+0000","last_fullsized":"2024-04-02T16:12:14.702801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707213+0000","last_change":"2024-04-02T16:12:14.707213+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707213+0000","last_undegraded":"2024-04-02T16:12:14.707213+0000","last_fullsized":"2024-04-02T16:12:14.707213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710836+0000","last_change":"2024-04-02T16:12:14.710836+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710836+0000","last_undegraded":"2024-04-02T16:12:14.710836+0000","last_fullsized":"2024-04-02T16:12:14.710836+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.718207+0000","last_change":"2024-04-02T16:12:14.718207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.718207+0000","last_undegraded":"2024-04-02T16:12:14.718207+0000","last_fullsized":"2024-04-02T16:12:14.718207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699274+0000","last_change":"2024-04-02T16:12:14.699274+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699274+0000","last_undegraded":"2024-04-02T16:12:14.699274+0000","last_fullsized":"2024-04-02T16:12:14.699274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.738979+0000","last_change":"2024-04-02T16:12:14.733795+0000","last_active":"2024-04-02T16:12:14.738979+0000","last_peered":"2024-04-02T16:12:14.738979+0000","last_clean":"2024-04-02T16:12:14.738979+0000","last_became_active":"2024-04-02T16:12:14.732992+0000","last_became_peered":"2024-04-02T16:12:14.732992+0000","last_unstale":"2024-04-02T16:12:14.738979+0000","last_undegraded":"2024-04-02T16:12:14.738979+0000","last_fullsized":"2024-04-02T16:12:14.738979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712203+0000","last_change":"2024-04-02T16:12:14.712203+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712203+0000","last_undegraded":"2024-04-02T16:12:14.712203+0000","last_fullsized":"2024-04-02T16:12:14.712203+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737483+0000","last_change":"2024-04-02T16:12:14.735269+0000","last_active":"2024-04-02T16:12:14.737483+0000","last_peered":"2024-04-02T16:12:14.737483+0000","last_clean":"2024-04-02T16:12:14.737483+0000","last_became_active":"2024-04-02T16:12:14.734879+0000","last_became_peered":"2024-04-02T16:12:14.734879+0000","last_unstale":"2024-04-02T16:12:14.737483+0000","last_undegraded":"2024-04-02T16:12:14.737483+0000","last_fullsized":"2024-04-02T16:12:14.737483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703492+0000","last_change":"2024-04-02T16:12:14.703492+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703492+0000","last_undegraded":"2024-04-02T16:12:14.703492+0000","last_fullsized":"2024-04-02T16:12:14.703492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712365+0000","last_change":"2024-04-02T16:12:14.711385+0000","last_active":"2024-04-02T16:12:14.712365+0000","last_peered":"2024-04-02T16:12:14.712365+0000","last_clean":"2024-04-02T16:12:14.712365+0000","last_became_active":"2024-04-02T16:12:14.711031+0000","last_became_peered":"2024-04-02T16:12:14.711031+0000","last_unstale":"2024-04-02T16:12:14.712365+0000","last_undegraded":"2024-04-02T16:12:14.712365+0000","last_fullsized":"2024-04-02T16:12:14.712365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697881+0000","last_change":"2024-04-02T16:12:14.697881+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697881+0000","last_undegraded":"2024-04-02T16:12:14.697881+0000","last_fullsized":"2024-04-02T16:12:14.697881+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716213+0000","last_change":"2024-04-02T16:12:14.715075+0000","last_active":"2024-04-02T16:12:14.716213+0000","last_peered":"2024-04-02T16:12:14.716213+0000","last_clean":"2024-04-02T16:12:14.716213+0000","last_became_active":"2024-04-02T16:12:14.714602+0000","last_became_peered":"2024-04-02T16:12:14.714602+0000","last_unstale":"2024-04-02T16:12:14.716213+0000","last_undegraded":"2024-04-02T16:12:14.716213+0000","last_fullsized":"2024-04-02T16:12:14.716213+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:19.028049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706098+0000","last_change":"2024-04-02T16:12:14.706098+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706098+0000","last_undegraded":"2024-04-02T16:12:14.706098+0000","last_fullsized":"2024-04-02T16:12:14.706098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722220+0000","last_change":"2024-04-02T16:12:14.719120+0000","last_active":"2024-04-02T16:12:14.722220+0000","last_peered":"2024-04-02T16:12:14.722220+0000","last_clean":"2024-04-02T16:12:14.722220+0000","last_became_active":"2024-04-02T16:12:14.718359+0000","last_became_peered":"2024-04-02T16:12:14.718359+0000","last_unstale":"2024-04-02T16:12:14.722220+0000","last_undegraded":"2024-04-02T16:12:14.722220+0000","last_fullsized":"2024-04-02T16:12:14.722220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:59.701393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707982+0000","last_change":"2024-04-02T16:12:14.707982+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707982+0000","last_undegraded":"2024-04-02T16:12:14.707982+0000","last_fullsized":"2024-04-02T16:12:14.707982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722598+0000","last_change":"2024-04-02T16:12:14.718750+0000","last_active":"2024-04-02T16:12:14.722598+0000","last_peered":"2024-04-02T16:12:14.722598+0000","last_clean":"2024-04-02T16:12:14.722598+0000","last_became_active":"2024-04-02T16:12:14.718131+0000","last_became_peered":"2024-04-02T16:12:14.718131+0000","last_unstale":"2024-04-02T16:12:14.722598+0000","last_undegraded":"2024-04-02T16:12:14.722598+0000","last_fullsized":"2024-04-02T16:12:14.722598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:24.153685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.701650+0000","last_change":"2024-04-02T16:12:14.701650+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.701650+0000","last_undegraded":"2024-04-02T16:12:14.701650+0000","last_fullsized":"2024-04-02T16:12:14.701650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.714009+0000","last_change":"2024-04-02T16:12:14.712471+0000","last_active":"2024-04-02T16:12:14.714009+0000","last_peered":"2024-04-02T16:12:14.714009+0000","last_clean":"2024-04-02T16:12:14.714009+0000","last_became_active":"2024-04-02T16:12:14.711972+0000","last_became_peered":"2024-04-02T16:12:14.711972+0000","last_unstale":"2024-04-02T16:12:14.714009+0000","last_undegraded":"2024-04-02T16:12:14.714009+0000","last_fullsized":"2024-04-02T16:12:14.714009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699448+0000","last_change":"2024-04-02T16:12:14.699448+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699448+0000","last_undegraded":"2024-04-02T16:12:14.699448+0000","last_fullsized":"2024-04-02T16:12:14.699448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729945+0000","last_change":"2024-04-02T16:12:14.725124+0000","last_active":"2024-04-02T16:12:14.729945+0000","last_peered":"2024-04-02T16:12:14.729945+0000","last_clean":"2024-04-02T16:12:14.729945+0000","last_became_active":"2024-04-02T16:12:14.724472+0000","last_became_peered":"2024-04-02T16:12:14.724472+0000","last_unstale":"2024-04-02T16:12:14.729945+0000","last_undegraded":"2024-04-02T16:12:14.729945+0000","last_fullsized":"2024-04-02T16:12:14.729945+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:40.524179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706851+0000","last_change":"2024-04-02T16:12:14.706851+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706851+0000","last_undegraded":"2024-04-02T16:12:14.706851+0000","last_fullsized":"2024-04-02T16:12:14.706851+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737486+0000","last_change":"2024-04-02T16:12:14.735619+0000","last_active":"2024-04-02T16:12:14.737486+0000","last_peered":"2024-04-02T16:12:14.737486+0000","last_clean":"2024-04-02T16:12:14.737486+0000","last_became_active":"2024-04-02T16:12:14.735188+0000","last_became_peered":"2024-04-02T16:12:14.735188+0000","last_unstale":"2024-04-02T16:12:14.737486+0000","last_undegraded":"2024-04-02T16:12:14.737486+0000","last_fullsized":"2024-04-02T16:12:14.737486+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702818+0000","last_change":"2024-04-02T16:12:14.702818+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702818+0000","last_undegraded":"2024-04-02T16:12:14.702818+0000","last_fullsized":"2024-04-02T16:12:14.702818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712128+0000","last_change":"2024-04-02T16:12:14.708255+0000","last_active":"2024-04-02T16:12:14.712128+0000","last_peered":"2024-04-02T16:12:14.712128+0000","last_clean":"2024-04-02T16:12:14.712128+0000","last_became_active":"2024-04-02T16:12:14.707800+0000","last_became_peered":"2024-04-02T16:12:14.707800+0000","last_unstale":"2024-04-02T16:12:14.712128+0000","last_undegraded":"2024-04-02T16:12:14.712128+0000","last_fullsized":"2024-04-02T16:12:14.712128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698603+0000","last_change":"2024-04-02T16:12:14.698603+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698603+0000","last_undegraded":"2024-04-02T16:12:14.698603+0000","last_fullsized":"2024-04-02T16:12:14.698603+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740173+0000","last_change":"2024-04-02T16:12:14.737929+0000","last_active":"2024-04-02T16:12:14.740173+0000","last_peered":"2024-04-02T16:12:14.740173+0000","last_clean":"2024-04-02T16:12:14.740173+0000","last_became_active":"2024-04-02T16:12:14.737178+0000","last_became_peered":"2024-04-02T16:12:14.737178+0000","last_unstale":"2024-04-02T16:12:14.740173+0000","last_undegraded":"2024-04-02T16:12:14.740173+0000","last_fullsized":"2024-04-02T16:12:14.740173+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711895+0000","last_change":"2024-04-02T16:12:14.711895+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711895+0000","last_undegraded":"2024-04-02T16:12:14.711895+0000","last_fullsized":"2024-04-02T16:12:14.711895+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726706+0000","last_change":"2024-04-02T16:12:14.719497+0000","last_active":"2024-04-02T16:12:14.726706+0000","last_peered":"2024-04-02T16:12:14.726706+0000","last_clean":"2024-04-02T16:12:14.726706+0000","last_became_active":"2024-04-02T16:12:14.719166+0000","last_became_peered":"2024-04-02T16:12:14.719166+0000","last_unstale":"2024-04-02T16:12:14.726706+0000","last_undegraded":"2024-04-02T16:12:14.726706+0000","last_fullsized":"2024-04-02T16:12:14.726706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:24.931785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706719+0000","last_change":"2024-04-02T16:12:14.706719+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706719+0000","last_undegraded":"2024-04-02T16:12:14.706719+0000","last_fullsized":"2024-04-02T16:12:14.706719+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733827+0000","last_change":"2024-04-02T16:12:14.726985+0000","last_active":"2024-04-02T16:12:14.733827+0000","last_peered":"2024-04-02T16:12:14.733827+0000","last_clean":"2024-04-02T16:12:14.733827+0000","last_became_active":"2024-04-02T16:12:14.726281+0000","last_became_peered":"2024-04-02T16:12:14.726281+0000","last_unstale":"2024-04-02T16:12:14.733827+0000","last_undegraded":"2024-04-02T16:12:14.733827+0000","last_fullsized":"2024-04-02T16:12:14.733827+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:43.241626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703947+0000","last_change":"2024-04-02T16:12:14.703947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703947+0000","last_undegraded":"2024-04-02T16:12:14.703947+0000","last_fullsized":"2024-04-02T16:12:14.703947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729153+0000","last_change":"2024-04-02T16:12:14.724232+0000","last_active":"2024-04-02T16:12:14.729153+0000","last_peered":"2024-04-02T16:12:14.729153+0000","last_clean":"2024-04-02T16:12:14.729153+0000","last_became_active":"2024-04-02T16:12:14.723889+0000","last_became_peered":"2024-04-02T16:12:14.723889+0000","last_unstale":"2024-04-02T16:12:14.729153+0000","last_undegraded":"2024-04-02T16:12:14.729153+0000","last_fullsized":"2024-04-02T16:12:14.729153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705456+0000","last_change":"2024-04-02T16:12:14.705456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705456+0000","last_undegraded":"2024-04-02T16:12:14.705456+0000","last_fullsized":"2024-04-02T16:12:14.705456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737546+0000","last_change":"2024-04-02T16:12:14.735767+0000","last_active":"2024-04-02T16:12:14.737546+0000","last_peered":"2024-04-02T16:12:14.737546+0000","last_clean":"2024-04-02T16:12:14.737546+0000","last_became_active":"2024-04-02T16:12:14.735499+0000","last_became_peered":"2024-04-02T16:12:14.735499+0000","last_unstale":"2024-04-02T16:12:14.737546+0000","last_undegraded":"2024-04-02T16:12:14.737546+0000","last_fullsized":"2024-04-02T16:12:14.737546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709624+0000","last_change":"2024-04-02T16:12:14.709624+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709624+0000","last_undegraded":"2024-04-02T16:12:14.709624+0000","last_fullsized":"2024-04-02T16:12:14.709624+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716043+0000","last_change":"2024-04-02T16:12:14.714641+0000","last_active":"2024-04-02T16:12:14.716043+0000","last_peered":"2024-04-02T16:12:14.716043+0000","last_clean":"2024-04-02T16:12:14.716043+0000","last_became_active":"2024-04-02T16:12:14.714242+0000","last_became_peered":"2024-04-02T16:12:14.714242+0000","last_unstale":"2024-04-02T16:12:14.716043+0000","last_undegraded":"2024-04-02T16:12:14.716043+0000","last_fullsized":"2024-04-02T16:12:14.716043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703317+0000","last_change":"2024-04-02T16:12:14.703317+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703317+0000","last_undegraded":"2024-04-02T16:12:14.703317+0000","last_fullsized":"2024-04-02T16:12:14.703317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.717058+0000","last_change":"2024-04-02T16:12:14.711749+0000","last_active":"2024-04-02T16:12:14.717058+0000","last_peered":"2024-04-02T16:12:14.717058+0000","last_clean":"2024-04-02T16:12:14.717058+0000","last_became_active":"2024-04-02T16:12:14.711308+0000","last_became_peered":"2024-04-02T16:12:14.711308+0000","last_unstale":"2024-04-02T16:12:14.717058+0000","last_undegraded":"2024-04-02T16:12:14.717058+0000","last_fullsized":"2024-04-02T16:12:14.717058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708089+0000","last_change":"2024-04-02T16:12:14.708089+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708089+0000","last_undegraded":"2024-04-02T16:12:14.708089+0000","last_fullsized":"2024-04-02T16:12:14.708089+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734463+0000","last_change":"2024-04-02T16:12:14.728114+0000","last_active":"2024-04-02T16:12:14.734463+0000","last_peered":"2024-04-02T16:12:14.734463+0000","last_clean":"2024-04-02T16:12:14.734463+0000","last_became_active":"2024-04-02T16:12:14.727429+0000","last_became_peered":"2024-04-02T16:12:14.727429+0000","last_unstale":"2024-04-02T16:12:14.734463+0000","last_undegraded":"2024-04-02T16:12:14.734463+0000","last_fullsized":"2024-04-02T16:12:14.734463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:05.668055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709940+0000","last_change":"2024-04-02T16:12:14.709940+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709940+0000","last_undegraded":"2024-04-02T16:12:14.709940+0000","last_fullsized":"2024-04-02T16:12:14.709940+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722851+0000","last_change":"2024-04-02T16:12:14.716172+0000","last_active":"2024-04-02T16:12:14.722851+0000","last_peered":"2024-04-02T16:12:14.722851+0000","last_clean":"2024-04-02T16:12:14.722851+0000","last_became_active":"2024-04-02T16:12:14.714967+0000","last_became_peered":"2024-04-02T16:12:14.714967+0000","last_unstale":"2024-04-02T16:12:14.722851+0000","last_undegraded":"2024-04-02T16:12:14.722851+0000","last_fullsized":"2024-04-02T16:12:14.722851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:19.185664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697302+0000","last_change":"2024-04-02T16:12:14.697302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697302+0000","last_undegraded":"2024-04-02T16:12:14.697302+0000","last_fullsized":"2024-04-02T16:12:14.697302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722096+0000","last_change":"2024-04-02T16:12:14.718370+0000","last_active":"2024-04-02T16:12:14.722096+0000","last_peered":"2024-04-02T16:12:14.722096+0000","last_clean":"2024-04-02T16:12:14.722096+0000","last_became_active":"2024-04-02T16:12:14.717749+0000","last_became_peered":"2024-04-02T16:12:14.717749+0000","last_unstale":"2024-04-02T16:12:14.722096+0000","last_undegraded":"2024-04-02T16:12:14.722096+0000","last_fullsized":"2024-04-02T16:12:14.722096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:19:34.428530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697456+0000","last_change":"2024-04-02T16:12:14.697456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697456+0000","last_undegraded":"2024-04-02T16:12:14.697456+0000","last_fullsized":"2024-04-02T16:12:14.697456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718780+0000","last_change":"2024-04-02T16:12:14.717818+0000","last_active":"2024-04-02T16:12:14.718780+0000","last_peered":"2024-04-02T16:12:14.718780+0000","last_clean":"2024-04-02T16:12:14.718780+0000","last_became_active":"2024-04-02T16:12:14.717461+0000","last_became_peered":"2024-04-02T16:12:14.717461+0000","last_unstale":"2024-04-02T16:12:14.718780+0000","last_undegraded":"2024-04-02T16:12:14.718780+0000","last_fullsized":"2024-04-02T16:12:14.718780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697306+0000","last_change":"2024-04-02T16:12:14.697306+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697306+0000","last_undegraded":"2024-04-02T16:12:14.697306+0000","last_fullsized":"2024-04-02T16:12:14.697306+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734053+0000","last_change":"2024-04-02T16:12:14.729387+0000","last_active":"2024-04-02T16:12:14.734053+0000","last_peered":"2024-04-02T16:12:14.734053+0000","last_clean":"2024-04-02T16:12:14.734053+0000","last_became_active":"2024-04-02T16:12:14.728744+0000","last_became_peered":"2024-04-02T16:12:14.728744+0000","last_unstale":"2024-04-02T16:12:14.734053+0000","last_undegraded":"2024-04-02T16:12:14.734053+0000","last_fullsized":"2024-04-02T16:12:14.734053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697542+0000","last_change":"2024-04-02T16:12:14.697542+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697542+0000","last_undegraded":"2024-04-02T16:12:14.697542+0000","last_fullsized":"2024-04-02T16:12:14.697542+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733963+0000","last_change":"2024-04-02T16:12:14.728831+0000","last_active":"2024-04-02T16:12:14.733963+0000","last_peered":"2024-04-02T16:12:14.733963+0000","last_clean":"2024-04-02T16:12:14.733963+0000","last_became_active":"2024-04-02T16:12:14.727741+0000","last_became_peered":"2024-04-02T16:12:14.727741+0000","last_unstale":"2024-04-02T16:12:14.733963+0000","last_undegraded":"2024-04-02T16:12:14.733963+0000","last_fullsized":"2024-04-02T16:12:14.733963+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:51.046626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710015+0000","last_change":"2024-04-02T16:12:14.710015+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710015+0000","last_undegraded":"2024-04-02T16:12:14.710015+0000","last_fullsized":"2024-04-02T16:12:14.710015+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740130+0000","last_change":"2024-04-02T16:12:14.737907+0000","last_active":"2024-04-02T16:12:14.740130+0000","last_peered":"2024-04-02T16:12:14.740130+0000","last_clean":"2024-04-02T16:12:14.740130+0000","last_became_active":"2024-04-02T16:12:14.737140+0000","last_became_peered":"2024-04-02T16:12:14.737140+0000","last_unstale":"2024-04-02T16:12:14.740130+0000","last_undegraded":"2024-04-02T16:12:14.740130+0000","last_fullsized":"2024-04-02T16:12:14.740130+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697201+0000","last_change":"2024-04-02T16:12:14.697201+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697201+0000","last_undegraded":"2024-04-02T16:12:14.697201+0000","last_fullsized":"2024-04-02T16:12:14.697201+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729195+0000","last_change":"2024-04-02T16:12:14.725690+0000","last_active":"2024-04-02T16:12:14.729195+0000","last_peered":"2024-04-02T16:12:14.729195+0000","last_clean":"2024-04-02T16:12:14.729195+0000","last_became_active":"2024-04-02T16:12:14.725425+0000","last_became_peered":"2024-04-02T16:12:14.725425+0000","last_unstale":"2024-04-02T16:12:14.729195+0000","last_undegraded":"2024-04-02T16:12:14.729195+0000","last_fullsized":"2024-04-02T16:12:14.729195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706947+0000","last_change":"2024-04-02T16:12:14.706947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706947+0000","last_undegraded":"2024-04-02T16:12:14.706947+0000","last_fullsized":"2024-04-02T16:12:14.706947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.713765+0000","last_change":"2024-04-02T16:12:14.711273+0000","last_active":"2024-04-02T16:12:14.713765+0000","last_peered":"2024-04-02T16:12:14.713765+0000","last_clean":"2024-04-02T16:12:14.713765+0000","last_became_active":"2024-04-02T16:12:14.710880+0000","last_became_peered":"2024-04-02T16:12:14.710880+0000","last_unstale":"2024-04-02T16:12:14.713765+0000","last_undegraded":"2024-04-02T16:12:14.713765+0000","last_fullsized":"2024-04-02T16:12:14.713765+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700493+0000","last_change":"2024-04-02T16:12:14.700493+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700493+0000","last_undegraded":"2024-04-02T16:12:14.700493+0000","last_fullsized":"2024-04-02T16:12:14.700493+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718705+0000","last_change":"2024-04-02T16:12:14.716204+0000","last_active":"2024-04-02T16:12:14.718705+0000","last_peered":"2024-04-02T16:12:14.718705+0000","last_clean":"2024-04-02T16:12:14.718705+0000","last_became_active":"2024-04-02T16:12:14.715693+0000","last_became_peered":"2024-04-02T16:12:14.715693+0000","last_unstale":"2024-04-02T16:12:14.718705+0000","last_undegraded":"2024-04-02T16:12:14.718705+0000","last_fullsized":"2024-04-02T16:12:14.718705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711442+0000","last_change":"2024-04-02T16:12:14.711442+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711442+0000","last_undegraded":"2024-04-02T16:12:14.711442+0000","last_fullsized":"2024-04-02T16:12:14.711442+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726471+0000","last_change":"2024-04-02T16:12:14.723590+0000","last_active":"2024-04-02T16:12:14.726471+0000","last_peered":"2024-04-02T16:12:14.726471+0000","last_clean":"2024-04-02T16:12:14.726471+0000","last_became_active":"2024-04-02T16:12:14.723145+0000","last_became_peered":"2024-04-02T16:12:14.723145+0000","last_unstale":"2024-04-02T16:12:14.726471+0000","last_undegraded":"2024-04-02T16:12:14.726471+0000","last_fullsized":"2024-04-02T16:12:14.726471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703745+0000","last_change":"2024-04-02T16:12:14.703745+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703745+0000","last_undegraded":"2024-04-02T16:12:14.703745+0000","last_fullsized":"2024-04-02T16:12:14.703745+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729200+0000","last_change":"2024-04-02T16:12:14.725705+0000","last_active":"2024-04-02T16:12:14.729200+0000","last_peered":"2024-04-02T16:12:14.729200+0000","last_clean":"2024-04-02T16:12:14.729200+0000","last_became_active":"2024-04-02T16:12:14.725450+0000","last_became_peered":"2024-04-02T16:12:14.725450+0000","last_unstale":"2024-04-02T16:12:14.729200+0000","last_undegraded":"2024-04-02T16:12:14.729200+0000","last_fullsized":"2024-04-02T16:12:14.729200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702914+0000","last_change":"2024-04-02T16:12:14.702914+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702914+0000","last_undegraded":"2024-04-02T16:12:14.702914+0000","last_fullsized":"2024-04-02T16:12:14.702914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734476+0000","last_change":"2024-04-02T16:12:14.726690+0000","last_active":"2024-04-02T16:12:14.734476+0000","last_peered":"2024-04-02T16:12:14.734476+0000","last_clean":"2024-04-02T16:12:14.734476+0000","last_became_active":"2024-04-02T16:12:14.725905+0000","last_became_peered":"2024-04-02T16:12:14.725905+0000","last_unstale":"2024-04-02T16:12:14.734476+0000","last_undegraded":"2024-04-02T16:12:14.734476+0000","last_fullsized":"2024-04-02T16:12:14.734476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:54.536550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704293+0000","last_change":"2024-04-02T16:12:14.704293+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704293+0000","last_undegraded":"2024-04-02T16:12:14.704293+0000","last_fullsized":"2024-04-02T16:12:14.704293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737552+0000","last_change":"2024-04-02T16:12:14.736388+0000","last_active":"2024-04-02T16:12:14.737552+0000","last_peered":"2024-04-02T16:12:14.737552+0000","last_clean":"2024-04-02T16:12:14.737552+0000","last_became_active":"2024-04-02T16:12:14.736096+0000","last_became_peered":"2024-04-02T16:12:14.736096+0000","last_unstale":"2024-04-02T16:12:14.737552+0000","last_undegraded":"2024-04-02T16:12:14.737552+0000","last_fullsized":"2024-04-02T16:12:14.737552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705554+0000","last_change":"2024-04-02T16:12:14.705554+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705554+0000","last_undegraded":"2024-04-02T16:12:14.705554+0000","last_fullsized":"2024-04-02T16:12:14.705554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737786+0000","last_change":"2024-04-02T16:12:14.735597+0000","last_active":"2024-04-02T16:12:14.737786+0000","last_peered":"2024-04-02T16:12:14.737786+0000","last_clean":"2024-04-02T16:12:14.737786+0000","last_became_active":"2024-04-02T16:12:14.734832+0000","last_became_peered":"2024-04-02T16:12:14.734832+0000","last_unstale":"2024-04-02T16:12:14.737786+0000","last_undegraded":"2024-04-02T16:12:14.737786+0000","last_fullsized":"2024-04-02T16:12:14.737786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.720269+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:14.720269+0000","last_peered":"2024-04-02T16:12:14.720269+0000","last_clean":"2024-04-02T16:12:14.720269+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:14.720269+0000","last_undegraded":"2024-04-02T16:12:14.720269+0000","last_fullsized":"2024-04-02T16:12:14.720269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:18.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:18.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 Processor -- start 2024-04-02T16:12:18.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 -- start start 2024-04-02T16:12:18.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5c413f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c40b26a0 con 0x7fa5c40b0b70 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c40730f0 con 0x7fa5c4072d10 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5cc2b1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c40b2a10 con 0x7fa5c40b40e0 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5ca84e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa5c40b0b70 0x7fa5c414fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55912/0 (socket says 172.21.15.40:55912) 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5ca84e640 1 -- 172.21.15.40:0/484049973 learned_addr learned my addr 172.21.15.40:0/484049973 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5c413f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314294317 0 0) 0x7fa5c40b26a0 con 0x7fa5c40b0b70 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5b0003530 con 0x7fa5c40b0b70 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 532234600 0 0) 0x7fa5b0003530 con 0x7fa5c40b0b70 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 msgr2=0x7fa5c413f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5c413f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 >> v1:172.21.15.204:6789/0 conn(0x7fa5c4072d10 legacy=0x7fa5c40b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c409fe40 con 0x7fa5c40b0b70 2024-04-02T16:12:18.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fa5bc001a00 con 0x7fa5c40b0b70 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 >> v1:172.21.15.40:6789/0 conn(0x7fa5c40b0b70 legacy=0x7fa5c414fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5b00039f0 0x7fa5b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c40b2a10 con 0x7fa5b0004810 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b000ba80 con 0x7fa5b00039f0 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b000bc80 con 0x7fa5c40b40e0 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5ca04d640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5b00039f0 0x7fa5b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5ca84e640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 -- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 msgr2=0x7fa5b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 -- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5b00039f0 msgr2=0x7fa5b0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5b00039f0 0x7fa5b0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5c984c640 1 -- 172.21.15.40:0/484049973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b0003530 con 0x7fa5b0004810 2024-04-02T16:12:18.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.912+0000 7fa5ca84e640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c40b40e0 0x7fa5b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5c984c640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5b0004c90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b7a10 tx=0x7fa5b401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67d3f7af7b3a6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40b2070 con 0x7fa5b0004810 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b40a55e0 con 0x7fa5b0004810 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5c904b640 1 -- 172.21.15.40:0/484049973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40096b0 con 0x7fa5b0004810 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/484049973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 msgr2=0x7fa5b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/484049973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5b0004c90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b7a10 tx=0x7fa5b401c9f0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5ca84e640 1 -- 172.21.15.40:0/484049973 reap_dead start 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/484049973 shutdown_connections 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/484049973 >> 172.21.15.40:0/484049973 conn(0x7fa5c40b7f50 msgr2=0x7fa5c40b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/484049973 shutdown_connections 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/484049973 wait complete. 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 Processor -- start 2024-04-02T16:12:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.916+0000 7fa5cc2b1640 1 -- start start 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5c4149050 0x7fa5c409f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c4071ae0 0x7fa5c4071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c40a0010 con 0x7fa5b0004810 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4145900 con 0x7fa5c4149050 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c413f4d0 con 0x7fa5c4071ae0 2024-04-02T16:12:18.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59252/0 (socket says 172.21.15.40:59252) 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 -- 172.21.15.40:0/539252288 learned_addr learned my addr 172.21.15.40:0/539252288 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca84e640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c4071ae0 0x7fa5c4071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5c984c640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5c4149050 0x7fa5c409f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 -- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c4071ae0 msgr2=0x7fa5c4071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c4071ae0 0x7fa5c4071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 -- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5c4149050 msgr2=0x7fa5c409f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5c4149050 0x7fa5c409f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 -- 172.21.15.40:0/539252288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c4148570 con 0x7fa5b0004810 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5ca04d640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e580 tx=0x7fa5c000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00052e0 con 0x7fa5b0004810 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c0018070 con 0x7fa5b0004810 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41469a0 con 0x7fa5b0004810 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003820 con 0x7fa5b0004810 2024-04-02T16:12:18.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c413e350 con 0x7fa5b0004810 2024-04-02T16:12:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4146790 con 0x7fa5b0004810 2024-04-02T16:12:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fa5c00039c0 con 0x7fa5b0004810 2024-04-02T16:12:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 0x7fa5a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fa5c0086710 con 0x7fa5b0004810 2024-04-02T16:12:18.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.920+0000 7fa5c984c640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 0x7fa5a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:18.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.924+0000 7fa5c984c640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 0x7fa5a806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40af000 tx=0x7fa5b40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:18.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:18.924+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fa5c0056ce0 con 0x7fa5b0004810 2024-04-02T16:12:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.092+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5c4068d00 con 0x7fa5a8068380 2024-04-02T16:12:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.108+0000 7fa5baffd640 1 -- 172.21.15.40:0/539252288 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fa5c4068d00 con 0x7fa5a8068380 2024-04-02T16:12:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 msgr2=0x7fa5a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 0x7fa5a806a840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40af000 tx=0x7fa5b40046d0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 msgr2=0x7fa5c41dff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e580 tx=0x7fa5c000ea40 comp rx=0 tx=0).stop 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 shutdown_connections 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa5c4071ae0 0x7fa5c4071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fa5a8068380 0x7fa5a806a840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5c4149050 0x7fa5c409f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 --2- 172.21.15.40:0/539252288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b0004810 0x7fa5c41dff20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 >> 172.21.15.40:0/539252288 conn(0x7fa5c40b7f50 msgr2=0x7fa5c4145ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 shutdown_connections 2024-04-02T16:12:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.112+0000 7fa5cc2b1640 1 -- 172.21.15.40:0/539252288 wait complete. 2024-04-02T16:12:19.132 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-04-02T16:12:17.863375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219596,"kb_used_data":3436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755124,"statfs":{"total":773094113280,"available":772869246976,"internally_reserved":0,"allocated":3518464,"data_stored":2036897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.181773"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.713236+0000","last_change":"2024-04-02T16:12:14.713236+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.713236+0000","last_undegraded":"2024-04-02T16:12:14.713236+0000","last_fullsized":"2024-04-02T16:12:14.713236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703207+0000","last_change":"2024-04-02T16:12:14.703207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703207+0000","last_undegraded":"2024-04-02T16:12:14.703207+0000","last_fullsized":"2024-04-02T16:12:14.703207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710853+0000","last_change":"2024-04-02T16:12:14.710853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710853+0000","last_undegraded":"2024-04-02T16:12:14.710853+0000","last_fullsized":"2024-04-02T16:12:14.710853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712715+0000","last_change":"2024-04-02T16:12:14.712715+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712715+0000","last_undegraded":"2024-04-02T16:12:14.712715+0000","last_fullsized":"2024-04-02T16:12:14.712715+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699530+0000","last_change":"2024-04-02T16:12:14.699530+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699530+0000","last_undegraded":"2024-04-02T16:12:14.699530+0000","last_fullsized":"2024-04-02T16:12:14.699530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698314+0000","last_change":"2024-04-02T16:12:14.698314+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698314+0000","last_undegraded":"2024-04-02T16:12:14.698314+0000","last_fullsized":"2024-04-02T16:12:14.698314+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704155+0000","last_change":"2024-04-02T16:12:14.704155+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704155+0000","last_undegraded":"2024-04-02T16:12:14.704155+0000","last_fullsized":"2024-04-02T16:12:14.704155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.719220+0000","last_change":"2024-04-02T16:12:14.719220+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.719220+0000","last_undegraded":"2024-04-02T16:12:14.719220+0000","last_fullsized":"2024-04-02T16:12:14.719220+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699751+0000","last_change":"2024-04-02T16:12:14.699751+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699751+0000","last_undegraded":"2024-04-02T16:12:14.699751+0000","last_fullsized":"2024-04-02T16:12:14.699751+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705979+0000","last_change":"2024-04-02T16:12:14.705979+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705979+0000","last_undegraded":"2024-04-02T16:12:14.705979+0000","last_fullsized":"2024-04-02T16:12:14.705979+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708302+0000","last_change":"2024-04-02T16:12:14.708302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708302+0000","last_undegraded":"2024-04-02T16:12:14.708302+0000","last_fullsized":"2024-04-02T16:12:14.708302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707900+0000","last_change":"2024-04-02T16:12:14.707900+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707900+0000","last_undegraded":"2024-04-02T16:12:14.707900+0000","last_fullsized":"2024-04-02T16:12:14.707900+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698803+0000","last_change":"2024-04-02T16:12:14.698803+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698803+0000","last_undegraded":"2024-04-02T16:12:14.698803+0000","last_fullsized":"2024-04-02T16:12:14.698803+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697784+0000","last_change":"2024-04-02T16:12:14.697784+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697784+0000","last_undegraded":"2024-04-02T16:12:14.697784+0000","last_fullsized":"2024-04-02T16:12:14.697784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710747+0000","last_change":"2024-04-02T16:12:14.710747+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710747+0000","last_undegraded":"2024-04-02T16:12:14.710747+0000","last_fullsized":"2024-04-02T16:12:14.710747+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704126+0000","last_change":"2024-04-02T16:12:14.704126+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704126+0000","last_undegraded":"2024-04-02T16:12:14.704126+0000","last_fullsized":"2024-04-02T16:12:14.704126+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706721+0000","last_change":"2024-04-02T16:12:14.706721+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706721+0000","last_undegraded":"2024-04-02T16:12:14.706721+0000","last_fullsized":"2024-04-02T16:12:14.706721+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702833+0000","last_change":"2024-04-02T16:12:14.702833+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702833+0000","last_undegraded":"2024-04-02T16:12:14.702833+0000","last_fullsized":"2024-04-02T16:12:14.702833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702324+0000","last_change":"2024-04-02T16:12:14.702324+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702324+0000","last_undegraded":"2024-04-02T16:12:14.702324+0000","last_fullsized":"2024-04-02T16:12:14.702324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697853+0000","last_change":"2024-04-02T16:12:14.697853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697853+0000","last_undegraded":"2024-04-02T16:12:14.697853+0000","last_fullsized":"2024-04-02T16:12:14.697853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703930+0000","last_change":"2024-04-02T16:12:14.703930+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703930+0000","last_undegraded":"2024-04-02T16:12:14.703930+0000","last_fullsized":"2024-04-02T16:12:14.703930+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700364+0000","last_change":"2024-04-02T16:12:14.700364+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700364+0000","last_undegraded":"2024-04-02T16:12:14.700364+0000","last_fullsized":"2024-04-02T16:12:14.700364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699911+0000","last_change":"2024-04-02T16:12:14.699911+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699911+0000","last_undegraded":"2024-04-02T16:12:14.699911+0000","last_fullsized":"2024-04-02T16:12:14.699911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705612+0000","last_change":"2024-04-02T16:12:14.705612+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705612+0000","last_undegraded":"2024-04-02T16:12:14.705612+0000","last_fullsized":"2024-04-02T16:12:14.705612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700630+0000","last_change":"2024-04-02T16:12:14.700630+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700630+0000","last_undegraded":"2024-04-02T16:12:14.700630+0000","last_fullsized":"2024-04-02T16:12:14.700630+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698892+0000","last_change":"2024-04-02T16:12:14.698892+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698892+0000","last_undegraded":"2024-04-02T16:12:14.698892+0000","last_fullsized":"2024-04-02T16:12:14.698892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698461+0000","last_change":"2024-04-02T16:12:14.698461+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698461+0000","last_undegraded":"2024-04-02T16:12:14.698461+0000","last_fullsized":"2024-04-02T16:12:14.698461+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706731+0000","last_change":"2024-04-02T16:12:14.706731+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706731+0000","last_undegraded":"2024-04-02T16:12:14.706731+0000","last_fullsized":"2024-04-02T16:12:14.706731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702801+0000","last_change":"2024-04-02T16:12:14.702801+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702801+0000","last_undegraded":"2024-04-02T16:12:14.702801+0000","last_fullsized":"2024-04-02T16:12:14.702801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707213+0000","last_change":"2024-04-02T16:12:14.707213+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707213+0000","last_undegraded":"2024-04-02T16:12:14.707213+0000","last_fullsized":"2024-04-02T16:12:14.707213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710836+0000","last_change":"2024-04-02T16:12:14.710836+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710836+0000","last_undegraded":"2024-04-02T16:12:14.710836+0000","last_fullsized":"2024-04-02T16:12:14.710836+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.718207+0000","last_change":"2024-04-02T16:12:14.718207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.718207+0000","last_undegraded":"2024-04-02T16:12:14.718207+0000","last_fullsized":"2024-04-02T16:12:14.718207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699274+0000","last_change":"2024-04-02T16:12:14.699274+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699274+0000","last_undegraded":"2024-04-02T16:12:14.699274+0000","last_fullsized":"2024-04-02T16:12:14.699274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.738979+0000","last_change":"2024-04-02T16:12:14.733795+0000","last_active":"2024-04-02T16:12:14.738979+0000","last_peered":"2024-04-02T16:12:14.738979+0000","last_clean":"2024-04-02T16:12:14.738979+0000","last_became_active":"2024-04-02T16:12:14.732992+0000","last_became_peered":"2024-04-02T16:12:14.732992+0000","last_unstale":"2024-04-02T16:12:14.738979+0000","last_undegraded":"2024-04-02T16:12:14.738979+0000","last_fullsized":"2024-04-02T16:12:14.738979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712203+0000","last_change":"2024-04-02T16:12:14.712203+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712203+0000","last_undegraded":"2024-04-02T16:12:14.712203+0000","last_fullsized":"2024-04-02T16:12:14.712203+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737483+0000","last_change":"2024-04-02T16:12:14.735269+0000","last_active":"2024-04-02T16:12:14.737483+0000","last_peered":"2024-04-02T16:12:14.737483+0000","last_clean":"2024-04-02T16:12:14.737483+0000","last_became_active":"2024-04-02T16:12:14.734879+0000","last_became_peered":"2024-04-02T16:12:14.734879+0000","last_unstale":"2024-04-02T16:12:14.737483+0000","last_undegraded":"2024-04-02T16:12:14.737483+0000","last_fullsized":"2024-04-02T16:12:14.737483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703492+0000","last_change":"2024-04-02T16:12:14.703492+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703492+0000","last_undegraded":"2024-04-02T16:12:14.703492+0000","last_fullsized":"2024-04-02T16:12:14.703492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712365+0000","last_change":"2024-04-02T16:12:14.711385+0000","last_active":"2024-04-02T16:12:14.712365+0000","last_peered":"2024-04-02T16:12:14.712365+0000","last_clean":"2024-04-02T16:12:14.712365+0000","last_became_active":"2024-04-02T16:12:14.711031+0000","last_became_peered":"2024-04-02T16:12:14.711031+0000","last_unstale":"2024-04-02T16:12:14.712365+0000","last_undegraded":"2024-04-02T16:12:14.712365+0000","last_fullsized":"2024-04-02T16:12:14.712365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697881+0000","last_change":"2024-04-02T16:12:14.697881+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697881+0000","last_undegraded":"2024-04-02T16:12:14.697881+0000","last_fullsized":"2024-04-02T16:12:14.697881+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716213+0000","last_change":"2024-04-02T16:12:14.715075+0000","last_active":"2024-04-02T16:12:14.716213+0000","last_peered":"2024-04-02T16:12:14.716213+0000","last_clean":"2024-04-02T16:12:14.716213+0000","last_became_active":"2024-04-02T16:12:14.714602+0000","last_became_peered":"2024-04-02T16:12:14.714602+0000","last_unstale":"2024-04-02T16:12:14.716213+0000","last_undegraded":"2024-04-02T16:12:14.716213+0000","last_fullsized":"2024-04-02T16:12:14.716213+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:19.028049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706098+0000","last_change":"2024-04-02T16:12:14.706098+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706098+0000","last_undegraded":"2024-04-02T16:12:14.706098+0000","last_fullsized":"2024-04-02T16:12:14.706098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722220+0000","last_change":"2024-04-02T16:12:14.719120+0000","last_active":"2024-04-02T16:12:14.722220+0000","last_peered":"2024-04-02T16:12:14.722220+0000","last_clean":"2024-04-02T16:12:14.722220+0000","last_became_active":"2024-04-02T16:12:14.718359+0000","last_became_peered":"2024-04-02T16:12:14.718359+0000","last_unstale":"2024-04-02T16:12:14.722220+0000","last_undegraded":"2024-04-02T16:12:14.722220+0000","last_fullsized":"2024-04-02T16:12:14.722220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:59.701393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707982+0000","last_change":"2024-04-02T16:12:14.707982+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707982+0000","last_undegraded":"2024-04-02T16:12:14.707982+0000","last_fullsized":"2024-04-02T16:12:14.707982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722598+0000","last_change":"2024-04-02T16:12:14.718750+0000","last_active":"2024-04-02T16:12:14.722598+0000","last_peered":"2024-04-02T16:12:14.722598+0000","last_clean":"2024-04-02T16:12:14.722598+0000","last_became_active":"2024-04-02T16:12:14.718131+0000","last_became_peered":"2024-04-02T16:12:14.718131+0000","last_unstale":"2024-04-02T16:12:14.722598+0000","last_undegraded":"2024-04-02T16:12:14.722598+0000","last_fullsized":"2024-04-02T16:12:14.722598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:24.153685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.701650+0000","last_change":"2024-04-02T16:12:14.701650+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.701650+0000","last_undegraded":"2024-04-02T16:12:14.701650+0000","last_fullsized":"2024-04-02T16:12:14.701650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.714009+0000","last_change":"2024-04-02T16:12:14.712471+0000","last_active":"2024-04-02T16:12:14.714009+0000","last_peered":"2024-04-02T16:12:14.714009+0000","last_clean":"2024-04-02T16:12:14.714009+0000","last_became_active":"2024-04-02T16:12:14.711972+0000","last_became_peered":"2024-04-02T16:12:14.711972+0000","last_unstale":"2024-04-02T16:12:14.714009+0000","last_undegraded":"2024-04-02T16:12:14.714009+0000","last_fullsized":"2024-04-02T16:12:14.714009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699448+0000","last_change":"2024-04-02T16:12:14.699448+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699448+0000","last_undegraded":"2024-04-02T16:12:14.699448+0000","last_fullsized":"2024-04-02T16:12:14.699448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729945+0000","last_change":"2024-04-02T16:12:14.725124+0000","last_active":"2024-04-02T16:12:14.729945+0000","last_peered":"2024-04-02T16:12:14.729945+0000","last_clean":"2024-04-02T16:12:14.729945+0000","last_became_active":"2024-04-02T16:12:14.724472+0000","last_became_peered":"2024-04-02T16:12:14.724472+0000","last_unstale":"2024-04-02T16:12:14.729945+0000","last_undegraded":"2024-04-02T16:12:14.729945+0000","last_fullsized":"2024-04-02T16:12:14.729945+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:40.524179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706851+0000","last_change":"2024-04-02T16:12:14.706851+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706851+0000","last_undegraded":"2024-04-02T16:12:14.706851+0000","last_fullsized":"2024-04-02T16:12:14.706851+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737486+0000","last_change":"2024-04-02T16:12:14.735619+0000","last_active":"2024-04-02T16:12:14.737486+0000","last_peered":"2024-04-02T16:12:14.737486+0000","last_clean":"2024-04-02T16:12:14.737486+0000","last_became_active":"2024-04-02T16:12:14.735188+0000","last_became_peered":"2024-04-02T16:12:14.735188+0000","last_unstale":"2024-04-02T16:12:14.737486+0000","last_undegraded":"2024-04-02T16:12:14.737486+0000","last_fullsized":"2024-04-02T16:12:14.737486+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702818+0000","last_change":"2024-04-02T16:12:14.702818+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702818+0000","last_undegraded":"2024-04-02T16:12:14.702818+0000","last_fullsized":"2024-04-02T16:12:14.702818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712128+0000","last_change":"2024-04-02T16:12:14.708255+0000","last_active":"2024-04-02T16:12:14.712128+0000","last_peered":"2024-04-02T16:12:14.712128+0000","last_clean":"2024-04-02T16:12:14.712128+0000","last_became_active":"2024-04-02T16:12:14.707800+0000","last_became_peered":"2024-04-02T16:12:14.707800+0000","last_unstale":"2024-04-02T16:12:14.712128+0000","last_undegraded":"2024-04-02T16:12:14.712128+0000","last_fullsized":"2024-04-02T16:12:14.712128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698603+0000","last_change":"2024-04-02T16:12:14.698603+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698603+0000","last_undegraded":"2024-04-02T16:12:14.698603+0000","last_fullsized":"2024-04-02T16:12:14.698603+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740173+0000","last_change":"2024-04-02T16:12:14.737929+0000","last_active":"2024-04-02T16:12:14.740173+0000","last_peered":"2024-04-02T16:12:14.740173+0000","last_clean":"2024-04-02T16:12:14.740173+0000","last_became_active":"2024-04-02T16:12:14.737178+0000","last_became_peered":"2024-04-02T16:12:14.737178+0000","last_unstale":"2024-04-02T16:12:14.740173+0000","last_undegraded":"2024-04-02T16:12:14.740173+0000","last_fullsized":"2024-04-02T16:12:14.740173+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711895+0000","last_change":"2024-04-02T16:12:14.711895+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711895+0000","last_undegraded":"2024-04-02T16:12:14.711895+0000","last_fullsized":"2024-04-02T16:12:14.711895+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726706+0000","last_change":"2024-04-02T16:12:14.719497+0000","last_active":"2024-04-02T16:12:14.726706+0000","last_peered":"2024-04-02T16:12:14.726706+0000","last_clean":"2024-04-02T16:12:14.726706+0000","last_became_active":"2024-04-02T16:12:14.719166+0000","last_became_peered":"2024-04-02T16:12:14.719166+0000","last_unstale":"2024-04-02T16:12:14.726706+0000","last_undegraded":"2024-04-02T16:12:14.726706+0000","last_fullsized":"2024-04-02T16:12:14.726706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:24.931785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706719+0000","last_change":"2024-04-02T16:12:14.706719+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706719+0000","last_undegraded":"2024-04-02T16:12:14.706719+0000","last_fullsized":"2024-04-02T16:12:14.706719+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733827+0000","last_change":"2024-04-02T16:12:14.726985+0000","last_active":"2024-04-02T16:12:14.733827+0000","last_peered":"2024-04-02T16:12:14.733827+0000","last_clean":"2024-04-02T16:12:14.733827+0000","last_became_active":"2024-04-02T16:12:14.726281+0000","last_became_peered":"2024-04-02T16:12:14.726281+0000","last_unstale":"2024-04-02T16:12:14.733827+0000","last_undegraded":"2024-04-02T16:12:14.733827+0000","last_fullsized":"2024-04-02T16:12:14.733827+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:43.241626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703947+0000","last_change":"2024-04-02T16:12:14.703947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703947+0000","last_undegraded":"2024-04-02T16:12:14.703947+0000","last_fullsized":"2024-04-02T16:12:14.703947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729153+0000","last_change":"2024-04-02T16:12:14.724232+0000","last_active":"2024-04-02T16:12:14.729153+0000","last_peered":"2024-04-02T16:12:14.729153+0000","last_clean":"2024-04-02T16:12:14.729153+0000","last_became_active":"2024-04-02T16:12:14.723889+0000","last_became_peered":"2024-04-02T16:12:14.723889+0000","last_unstale":"2024-04-02T16:12:14.729153+0000","last_undegraded":"2024-04-02T16:12:14.729153+0000","last_fullsized":"2024-04-02T16:12:14.729153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705456+0000","last_change":"2024-04-02T16:12:14.705456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705456+0000","last_undegraded":"2024-04-02T16:12:14.705456+0000","last_fullsized":"2024-04-02T16:12:14.705456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737546+0000","last_change":"2024-04-02T16:12:14.735767+0000","last_active":"2024-04-02T16:12:14.737546+0000","last_peered":"2024-04-02T16:12:14.737546+0000","last_clean":"2024-04-02T16:12:14.737546+0000","last_became_active":"2024-04-02T16:12:14.735499+0000","last_became_peered":"2024-04-02T16:12:14.735499+0000","last_unstale":"2024-04-02T16:12:14.737546+0000","last_undegraded":"2024-04-02T16:12:14.737546+0000","last_fullsized":"2024-04-02T16:12:14.737546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709624+0000","last_change":"2024-04-02T16:12:14.709624+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709624+0000","last_undegraded":"2024-04-02T16:12:14.709624+0000","last_fullsized":"2024-04-02T16:12:14.709624+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716043+0000","last_change":"2024-04-02T16:12:14.714641+0000","last_active":"2024-04-02T16:12:14.716043+0000","last_peered":"2024-04-02T16:12:14.716043+0000","last_clean":"2024-04-02T16:12:14.716043+0000","last_became_active":"2024-04-02T16:12:14.714242+0000","last_became_peered":"2024-04-02T16:12:14.714242+0000","last_unstale":"2024-04-02T16:12:14.716043+0000","last_undegraded":"2024-04-02T16:12:14.716043+0000","last_fullsized":"2024-04-02T16:12:14.716043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703317+0000","last_change":"2024-04-02T16:12:14.703317+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703317+0000","last_undegraded":"2024-04-02T16:12:14.703317+0000","last_fullsized":"2024-04-02T16:12:14.703317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.717058+0000","last_change":"2024-04-02T16:12:14.711749+0000","last_active":"2024-04-02T16:12:14.717058+0000","last_peered":"2024-04-02T16:12:14.717058+0000","last_clean":"2024-04-02T16:12:14.717058+0000","last_became_active":"2024-04-02T16:12:14.711308+0000","last_became_peered":"2024-04-02T16:12:14.711308+0000","last_unstale":"2024-04-02T16:12:14.717058+0000","last_undegraded":"2024-04-02T16:12:14.717058+0000","last_fullsized":"2024-04-02T16:12:14.717058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708089+0000","last_change":"2024-04-02T16:12:14.708089+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708089+0000","last_undegraded":"2024-04-02T16:12:14.708089+0000","last_fullsized":"2024-04-02T16:12:14.708089+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734463+0000","last_change":"2024-04-02T16:12:14.728114+0000","last_active":"2024-04-02T16:12:14.734463+0000","last_peered":"2024-04-02T16:12:14.734463+0000","last_clean":"2024-04-02T16:12:14.734463+0000","last_became_active":"2024-04-02T16:12:14.727429+0000","last_became_peered":"2024-04-02T16:12:14.727429+0000","last_unstale":"2024-04-02T16:12:14.734463+0000","last_undegraded":"2024-04-02T16:12:14.734463+0000","last_fullsized":"2024-04-02T16:12:14.734463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:05.668055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709940+0000","last_change":"2024-04-02T16:12:14.709940+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709940+0000","last_undegraded":"2024-04-02T16:12:14.709940+0000","last_fullsized":"2024-04-02T16:12:14.709940+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722851+0000","last_change":"2024-04-02T16:12:14.716172+0000","last_active":"2024-04-02T16:12:14.722851+0000","last_peered":"2024-04-02T16:12:14.722851+0000","last_clean":"2024-04-02T16:12:14.722851+0000","last_became_active":"2024-04-02T16:12:14.714967+0000","last_became_peered":"2024-04-02T16:12:14.714967+0000","last_unstale":"2024-04-02T16:12:14.722851+0000","last_undegraded":"2024-04-02T16:12:14.722851+0000","last_fullsized":"2024-04-02T16:12:14.722851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:19.185664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697302+0000","last_change":"2024-04-02T16:12:14.697302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697302+0000","last_undegraded":"2024-04-02T16:12:14.697302+0000","last_fullsized":"2024-04-02T16:12:14.697302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722096+0000","last_change":"2024-04-02T16:12:14.718370+0000","last_active":"2024-04-02T16:12:14.722096+0000","last_peered":"2024-04-02T16:12:14.722096+0000","last_clean":"2024-04-02T16:12:14.722096+0000","last_became_active":"2024-04-02T16:12:14.717749+0000","last_became_peered":"2024-04-02T16:12:14.717749+0000","last_unstale":"2024-04-02T16:12:14.722096+0000","last_undegraded":"2024-04-02T16:12:14.722096+0000","last_fullsized":"2024-04-02T16:12:14.722096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:19:34.428530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697456+0000","last_change":"2024-04-02T16:12:14.697456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697456+0000","last_undegraded":"2024-04-02T16:12:14.697456+0000","last_fullsized":"2024-04-02T16:12:14.697456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718780+0000","last_change":"2024-04-02T16:12:14.717818+0000","last_active":"2024-04-02T16:12:14.718780+0000","last_peered":"2024-04-02T16:12:14.718780+0000","last_clean":"2024-04-02T16:12:14.718780+0000","last_became_active":"2024-04-02T16:12:14.717461+0000","last_became_peered":"2024-04-02T16:12:14.717461+0000","last_unstale":"2024-04-02T16:12:14.718780+0000","last_undegraded":"2024-04-02T16:12:14.718780+0000","last_fullsized":"2024-04-02T16:12:14.718780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697306+0000","last_change":"2024-04-02T16:12:14.697306+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697306+0000","last_undegraded":"2024-04-02T16:12:14.697306+0000","last_fullsized":"2024-04-02T16:12:14.697306+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734053+0000","last_change":"2024-04-02T16:12:14.729387+0000","last_active":"2024-04-02T16:12:14.734053+0000","last_peered":"2024-04-02T16:12:14.734053+0000","last_clean":"2024-04-02T16:12:14.734053+0000","last_became_active":"2024-04-02T16:12:14.728744+0000","last_became_peered":"2024-04-02T16:12:14.728744+0000","last_unstale":"2024-04-02T16:12:14.734053+0000","last_undegraded":"2024-04-02T16:12:14.734053+0000","last_fullsized":"2024-04-02T16:12:14.734053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697542+0000","last_change":"2024-04-02T16:12:14.697542+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697542+0000","last_undegraded":"2024-04-02T16:12:14.697542+0000","last_fullsized":"2024-04-02T16:12:14.697542+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733963+0000","last_change":"2024-04-02T16:12:14.728831+0000","last_active":"2024-04-02T16:12:14.733963+0000","last_peered":"2024-04-02T16:12:14.733963+0000","last_clean":"2024-04-02T16:12:14.733963+0000","last_became_active":"2024-04-02T16:12:14.727741+0000","last_became_peered":"2024-04-02T16:12:14.727741+0000","last_unstale":"2024-04-02T16:12:14.733963+0000","last_undegraded":"2024-04-02T16:12:14.733963+0000","last_fullsized":"2024-04-02T16:12:14.733963+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:51.046626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710015+0000","last_change":"2024-04-02T16:12:14.710015+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710015+0000","last_undegraded":"2024-04-02T16:12:14.710015+0000","last_fullsized":"2024-04-02T16:12:14.710015+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740130+0000","last_change":"2024-04-02T16:12:14.737907+0000","last_active":"2024-04-02T16:12:14.740130+0000","last_peered":"2024-04-02T16:12:14.740130+0000","last_clean":"2024-04-02T16:12:14.740130+0000","last_became_active":"2024-04-02T16:12:14.737140+0000","last_became_peered":"2024-04-02T16:12:14.737140+0000","last_unstale":"2024-04-02T16:12:14.740130+0000","last_undegraded":"2024-04-02T16:12:14.740130+0000","last_fullsized":"2024-04-02T16:12:14.740130+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697201+0000","last_change":"2024-04-02T16:12:14.697201+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697201+0000","last_undegraded":"2024-04-02T16:12:14.697201+0000","last_fullsized":"2024-04-02T16:12:14.697201+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729195+0000","last_change":"2024-04-02T16:12:14.725690+0000","last_active":"2024-04-02T16:12:14.729195+0000","last_peered":"2024-04-02T16:12:14.729195+0000","last_clean":"2024-04-02T16:12:14.729195+0000","last_became_active":"2024-04-02T16:12:14.725425+0000","last_became_peered":"2024-04-02T16:12:14.725425+0000","last_unstale":"2024-04-02T16:12:14.729195+0000","last_undegraded":"2024-04-02T16:12:14.729195+0000","last_fullsized":"2024-04-02T16:12:14.729195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706947+0000","last_change":"2024-04-02T16:12:14.706947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706947+0000","last_undegraded":"2024-04-02T16:12:14.706947+0000","last_fullsized":"2024-04-02T16:12:14.706947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.713765+0000","last_change":"2024-04-02T16:12:14.711273+0000","last_active":"2024-04-02T16:12:14.713765+0000","last_peered":"2024-04-02T16:12:14.713765+0000","last_clean":"2024-04-02T16:12:14.713765+0000","last_became_active":"2024-04-02T16:12:14.710880+0000","last_became_peered":"2024-04-02T16:12:14.710880+0000","last_unstale":"2024-04-02T16:12:14.713765+0000","last_undegraded":"2024-04-02T16:12:14.713765+0000","last_fullsized":"2024-04-02T16:12:14.713765+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700493+0000","last_change":"2024-04-02T16:12:14.700493+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700493+0000","last_undegraded":"2024-04-02T16:12:14.700493+0000","last_fullsized":"2024-04-02T16:12:14.700493+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718705+0000","last_change":"2024-04-02T16:12:14.716204+0000","last_active":"2024-04-02T16:12:14.718705+0000","last_peered":"2024-04-02T16:12:14.718705+0000","last_clean":"2024-04-02T16:12:14.718705+0000","last_became_active":"2024-04-02T16:12:14.715693+0000","last_became_peered":"2024-04-02T16:12:14.715693+0000","last_unstale":"2024-04-02T16:12:14.718705+0000","last_undegraded":"2024-04-02T16:12:14.718705+0000","last_fullsized":"2024-04-02T16:12:14.718705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711442+0000","last_change":"2024-04-02T16:12:14.711442+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711442+0000","last_undegraded":"2024-04-02T16:12:14.711442+0000","last_fullsized":"2024-04-02T16:12:14.711442+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726471+0000","last_change":"2024-04-02T16:12:14.723590+0000","last_active":"2024-04-02T16:12:14.726471+0000","last_peered":"2024-04-02T16:12:14.726471+0000","last_clean":"2024-04-02T16:12:14.726471+0000","last_became_active":"2024-04-02T16:12:14.723145+0000","last_became_peered":"2024-04-02T16:12:14.723145+0000","last_unstale":"2024-04-02T16:12:14.726471+0000","last_undegraded":"2024-04-02T16:12:14.726471+0000","last_fullsized":"2024-04-02T16:12:14.726471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703745+0000","last_change":"2024-04-02T16:12:14.703745+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703745+0000","last_undegraded":"2024-04-02T16:12:14.703745+0000","last_fullsized":"2024-04-02T16:12:14.703745+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729200+0000","last_change":"2024-04-02T16:12:14.725705+0000","last_active":"2024-04-02T16:12:14.729200+0000","last_peered":"2024-04-02T16:12:14.729200+0000","last_clean":"2024-04-02T16:12:14.729200+0000","last_became_active":"2024-04-02T16:12:14.725450+0000","last_became_peered":"2024-04-02T16:12:14.725450+0000","last_unstale":"2024-04-02T16:12:14.729200+0000","last_undegraded":"2024-04-02T16:12:14.729200+0000","last_fullsized":"2024-04-02T16:12:14.729200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702914+0000","last_change":"2024-04-02T16:12:14.702914+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702914+0000","last_undegraded":"2024-04-02T16:12:14.702914+0000","last_fullsized":"2024-04-02T16:12:14.702914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734476+0000","last_change":"2024-04-02T16:12:14.726690+0000","last_active":"2024-04-02T16:12:14.734476+0000","last_peered":"2024-04-02T16:12:14.734476+0000","last_clean":"2024-04-02T16:12:14.734476+0000","last_became_active":"2024-04-02T16:12:14.725905+0000","last_became_peered":"2024-04-02T16:12:14.725905+0000","last_unstale":"2024-04-02T16:12:14.734476+0000","last_undegraded":"2024-04-02T16:12:14.734476+0000","last_fullsized":"2024-04-02T16:12:14.734476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:54.536550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704293+0000","last_change":"2024-04-02T16:12:14.704293+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704293+0000","last_undegraded":"2024-04-02T16:12:14.704293+0000","last_fullsized":"2024-04-02T16:12:14.704293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737552+0000","last_change":"2024-04-02T16:12:14.736388+0000","last_active":"2024-04-02T16:12:14.737552+0000","last_peered":"2024-04-02T16:12:14.737552+0000","last_clean":"2024-04-02T16:12:14.737552+0000","last_became_active":"2024-04-02T16:12:14.736096+0000","last_became_peered":"2024-04-02T16:12:14.736096+0000","last_unstale":"2024-04-02T16:12:14.737552+0000","last_undegraded":"2024-04-02T16:12:14.737552+0000","last_fullsized":"2024-04-02T16:12:14.737552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705554+0000","last_change":"2024-04-02T16:12:14.705554+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705554+0000","last_undegraded":"2024-04-02T16:12:14.705554+0000","last_fullsized":"2024-04-02T16:12:14.705554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737786+0000","last_change":"2024-04-02T16:12:14.735597+0000","last_active":"2024-04-02T16:12:14.737786+0000","last_peered":"2024-04-02T16:12:14.737786+0000","last_clean":"2024-04-02T16:12:14.737786+0000","last_became_active":"2024-04-02T16:12:14.734832+0000","last_became_peered":"2024-04-02T16:12:14.734832+0000","last_unstale":"2024-04-02T16:12:14.737786+0000","last_undegraded":"2024-04-02T16:12:14.737786+0000","last_fullsized":"2024-04-02T16:12:14.737786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.720269+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:14.720269+0000","last_peered":"2024-04-02T16:12:14.720269+0000","last_clean":"2024-04-02T16:12:14.720269+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:14.720269+0000","last_undegraded":"2024-04-02T16:12:14.720269+0000","last_fullsized":"2024-04-02T16:12:14.720269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:19.136 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.296+0000 7f7868308640 1 Processor -- start 2024-04-02T16:12:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.296+0000 7f7868308640 1 -- start start 2024-04-02T16:12:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f7868308640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f786009f0e0 0x7f78600b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f7868308640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f786009f4c0 con 0x7f7860141b30 2024-04-02T16:12:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f7868308640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7860071a00 con 0x7f78600b4030 2024-04-02T16:12:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f7868308640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7860071220 con 0x7f786009f0e0 2024-04-02T16:12:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78668a5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7860141b30 0x7f7860152000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55918/0 (socket says 172.21.15.40:55918) 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78668a5640 1 -- 172.21.15.40:0/3929715184 learned_addr learned my addr 172.21.15.40:0/3929715184 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 --2- 172.21.15.40:0/3929715184 >> v2:172.21.15.204:3300/0 conn(0x7f786009f0e0 0x7f78600b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1515887843 0 0) 0x7f786009f4c0 con 0x7f7860141b30 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f784c003530 con 0x7f7860141b30 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3125998941 0 0) 0x7f784c003530 con 0x7f7860141b30 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 >> v2:172.21.15.204:3300/0 conn(0x7f786009f0e0 msgr2=0x7f78600b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 --2- 172.21.15.40:0/3929715184 >> v2:172.21.15.204:3300/0 conn(0x7f786009f0e0 0x7f78600b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 >> v1:172.21.15.204:6789/0 conn(0x7f78600b4030 legacy=0x7f7860141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7860071500 con 0x7f7860141b30 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f7858001680 con 0x7f7860141b30 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 >> v1:172.21.15.40:6789/0 conn(0x7f7860141b30 legacy=0x7f7860152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 0x7f784c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c004810 0x7f784c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784c005220 0x7f784c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7860071220 con 0x7f784c003a10 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f784c00c170 con 0x7f784c004810 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f784c00c370 con 0x7f784c005220 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 0x7f784c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78668a5640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784c005220 0x7f784c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78658a3640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c004810 0x7f784c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 -- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784c005220 msgr2=0x7f784c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f784c005220 0x7f784c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 -- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c004810 msgr2=0x7f784c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c004810 0x7f784c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 -- 172.21.15.40:0/3929715184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c003530 con 0x7f784c003a10 2024-04-02T16:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78668a5640 1 -- 172.21.15.40:0/3929715184 reap_dead start 2024-04-02T16:12:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78658a3640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c004810 0x7f784c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78660a4640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 0x7f784c003e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f785c009f90 tx=0x7f785c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833b1423aa644eea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0b2070 con 0x7f784c003a10 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785c0a5b40 con 0x7f784c003a10 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.300+0000 7f78650a2640 1 -- 172.21.15.40:0/3929715184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c009c10 con 0x7f784c003a10 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 msgr2=0x7f784c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 0x7f784c003e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f785c009f90 tx=0x7f785c01cc40 comp rx=0 tx=0).stop 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3929715184 shutdown_connections 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 --2- 172.21.15.40:0/3929715184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f784c003a10 0x7f784c003e30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3929715184 >> 172.21.15.40:0/3929715184 conn(0x7f78600b7f50 msgr2=0x7f78600b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3929715184 shutdown_connections 2024-04-02T16:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3929715184 wait complete. 2024-04-02T16:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 Processor -- start 2024-04-02T16:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- start start 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c003a10 0x7f786014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7860149530 0x7f78601e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78601419b0 con 0x7f7860149990 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7860141830 con 0x7f784c003a10 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78601476c0 con 0x7f7860149530 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59278/0 (socket says 172.21.15.40:59278) 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 -- 172.21.15.40:0/3021411687 learned_addr learned my addr 172.21.15.40:0/3021411687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78658a3640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7860149530 0x7f78601e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78660a4640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c003a10 0x7f786014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 -- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7860149530 msgr2=0x7f78601e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7860149530 0x7f78601e41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 -- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c003a10 msgr2=0x7f786014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c003a10 0x7f786014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 -- 172.21.15.40:0/3021411687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786014ab20 con 0x7f7860149990 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f78668a5640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f786002d0a0 tx=0x7f785800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78580074e0 con 0x7f7860149990 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78580047f0 con 0x7f7860149990 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78601450e0 con 0x7f7860149990 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858007680 con 0x7f7860149990 2024-04-02T16:12:19.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.304+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7860148960 con 0x7f7860149990 2024-04-02T16:12:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7860068d00 con 0x7f7860149990 2024-04-02T16:12:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f785801b020 con 0x7f7860149990 2024-04-02T16:12:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f7856ffd640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 0x7f784406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f7858086ba0 con 0x7f7860149990 2024-04-02T16:12:19.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f78660a4640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 0x7f784406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.308+0000 7f78660a4640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 0x7f784406a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f785c0af000 tx=0x7f785c0a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.312+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f7858022070 con 0x7f7860149990 2024-04-02T16:12:19.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.468+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f786009f4c0 con 0x7f7844068380 2024-04-02T16:12:19.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.484+0000 7f7856ffd640 1 -- 172.21.15.40:0/3021411687 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7f786009f4c0 con 0x7f7844068380 2024-04-02T16:12:19.490 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:19.491 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.488+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 msgr2=0x7f784406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.488+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 0x7f784406a840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f785c0af000 tx=0x7f785c0a7040 comp rx=0 tx=0).stop 2024-04-02T16:12:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.488+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 msgr2=0x7f78601e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.488+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f786002d0a0 tx=0x7f785800d9d0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 shutdown_connections 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7860149530 0x7f78601e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7844068380 0x7f784406a840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7860149990 0x7f78601e0510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 --2- 172.21.15.40:0/3021411687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784c003a10 0x7f786014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 >> 172.21.15.40:0/3021411687 conn(0x7f78600b7f50 msgr2=0x7f786013e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 shutdown_connections 2024-04-02T16:12:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.492+0000 7f7868308640 1 -- 172.21.15.40:0/3021411687 wait complete. 2024-04-02T16:12:19.508 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-04-02T16:12:17.863375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219596,"kb_used_data":3436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755124,"statfs":{"total":773094113280,"available":772869246976,"internally_reserved":0,"allocated":3518464,"data_stored":2036897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.181773"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.713236+0000","last_change":"2024-04-02T16:12:14.713236+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.713236+0000","last_undegraded":"2024-04-02T16:12:14.713236+0000","last_fullsized":"2024-04-02T16:12:14.713236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703207+0000","last_change":"2024-04-02T16:12:14.703207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703207+0000","last_undegraded":"2024-04-02T16:12:14.703207+0000","last_fullsized":"2024-04-02T16:12:14.703207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710853+0000","last_change":"2024-04-02T16:12:14.710853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710853+0000","last_undegraded":"2024-04-02T16:12:14.710853+0000","last_fullsized":"2024-04-02T16:12:14.710853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712715+0000","last_change":"2024-04-02T16:12:14.712715+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712715+0000","last_undegraded":"2024-04-02T16:12:14.712715+0000","last_fullsized":"2024-04-02T16:12:14.712715+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699530+0000","last_change":"2024-04-02T16:12:14.699530+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699530+0000","last_undegraded":"2024-04-02T16:12:14.699530+0000","last_fullsized":"2024-04-02T16:12:14.699530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698314+0000","last_change":"2024-04-02T16:12:14.698314+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698314+0000","last_undegraded":"2024-04-02T16:12:14.698314+0000","last_fullsized":"2024-04-02T16:12:14.698314+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704155+0000","last_change":"2024-04-02T16:12:14.704155+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704155+0000","last_undegraded":"2024-04-02T16:12:14.704155+0000","last_fullsized":"2024-04-02T16:12:14.704155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.719220+0000","last_change":"2024-04-02T16:12:14.719220+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.719220+0000","last_undegraded":"2024-04-02T16:12:14.719220+0000","last_fullsized":"2024-04-02T16:12:14.719220+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699751+0000","last_change":"2024-04-02T16:12:14.699751+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699751+0000","last_undegraded":"2024-04-02T16:12:14.699751+0000","last_fullsized":"2024-04-02T16:12:14.699751+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705979+0000","last_change":"2024-04-02T16:12:14.705979+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705979+0000","last_undegraded":"2024-04-02T16:12:14.705979+0000","last_fullsized":"2024-04-02T16:12:14.705979+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708302+0000","last_change":"2024-04-02T16:12:14.708302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708302+0000","last_undegraded":"2024-04-02T16:12:14.708302+0000","last_fullsized":"2024-04-02T16:12:14.708302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707900+0000","last_change":"2024-04-02T16:12:14.707900+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707900+0000","last_undegraded":"2024-04-02T16:12:14.707900+0000","last_fullsized":"2024-04-02T16:12:14.707900+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698803+0000","last_change":"2024-04-02T16:12:14.698803+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698803+0000","last_undegraded":"2024-04-02T16:12:14.698803+0000","last_fullsized":"2024-04-02T16:12:14.698803+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697784+0000","last_change":"2024-04-02T16:12:14.697784+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697784+0000","last_undegraded":"2024-04-02T16:12:14.697784+0000","last_fullsized":"2024-04-02T16:12:14.697784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710747+0000","last_change":"2024-04-02T16:12:14.710747+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710747+0000","last_undegraded":"2024-04-02T16:12:14.710747+0000","last_fullsized":"2024-04-02T16:12:14.710747+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704126+0000","last_change":"2024-04-02T16:12:14.704126+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704126+0000","last_undegraded":"2024-04-02T16:12:14.704126+0000","last_fullsized":"2024-04-02T16:12:14.704126+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706721+0000","last_change":"2024-04-02T16:12:14.706721+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706721+0000","last_undegraded":"2024-04-02T16:12:14.706721+0000","last_fullsized":"2024-04-02T16:12:14.706721+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702833+0000","last_change":"2024-04-02T16:12:14.702833+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702833+0000","last_undegraded":"2024-04-02T16:12:14.702833+0000","last_fullsized":"2024-04-02T16:12:14.702833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702324+0000","last_change":"2024-04-02T16:12:14.702324+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702324+0000","last_undegraded":"2024-04-02T16:12:14.702324+0000","last_fullsized":"2024-04-02T16:12:14.702324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697853+0000","last_change":"2024-04-02T16:12:14.697853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697853+0000","last_undegraded":"2024-04-02T16:12:14.697853+0000","last_fullsized":"2024-04-02T16:12:14.697853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703930+0000","last_change":"2024-04-02T16:12:14.703930+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703930+0000","last_undegraded":"2024-04-02T16:12:14.703930+0000","last_fullsized":"2024-04-02T16:12:14.703930+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700364+0000","last_change":"2024-04-02T16:12:14.700364+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700364+0000","last_undegraded":"2024-04-02T16:12:14.700364+0000","last_fullsized":"2024-04-02T16:12:14.700364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699911+0000","last_change":"2024-04-02T16:12:14.699911+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699911+0000","last_undegraded":"2024-04-02T16:12:14.699911+0000","last_fullsized":"2024-04-02T16:12:14.699911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705612+0000","last_change":"2024-04-02T16:12:14.705612+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705612+0000","last_undegraded":"2024-04-02T16:12:14.705612+0000","last_fullsized":"2024-04-02T16:12:14.705612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700630+0000","last_change":"2024-04-02T16:12:14.700630+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700630+0000","last_undegraded":"2024-04-02T16:12:14.700630+0000","last_fullsized":"2024-04-02T16:12:14.700630+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698892+0000","last_change":"2024-04-02T16:12:14.698892+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698892+0000","last_undegraded":"2024-04-02T16:12:14.698892+0000","last_fullsized":"2024-04-02T16:12:14.698892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698461+0000","last_change":"2024-04-02T16:12:14.698461+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698461+0000","last_undegraded":"2024-04-02T16:12:14.698461+0000","last_fullsized":"2024-04-02T16:12:14.698461+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706731+0000","last_change":"2024-04-02T16:12:14.706731+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706731+0000","last_undegraded":"2024-04-02T16:12:14.706731+0000","last_fullsized":"2024-04-02T16:12:14.706731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702801+0000","last_change":"2024-04-02T16:12:14.702801+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702801+0000","last_undegraded":"2024-04-02T16:12:14.702801+0000","last_fullsized":"2024-04-02T16:12:14.702801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707213+0000","last_change":"2024-04-02T16:12:14.707213+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707213+0000","last_undegraded":"2024-04-02T16:12:14.707213+0000","last_fullsized":"2024-04-02T16:12:14.707213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710836+0000","last_change":"2024-04-02T16:12:14.710836+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710836+0000","last_undegraded":"2024-04-02T16:12:14.710836+0000","last_fullsized":"2024-04-02T16:12:14.710836+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.718207+0000","last_change":"2024-04-02T16:12:14.718207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.718207+0000","last_undegraded":"2024-04-02T16:12:14.718207+0000","last_fullsized":"2024-04-02T16:12:14.718207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699274+0000","last_change":"2024-04-02T16:12:14.699274+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699274+0000","last_undegraded":"2024-04-02T16:12:14.699274+0000","last_fullsized":"2024-04-02T16:12:14.699274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.738979+0000","last_change":"2024-04-02T16:12:14.733795+0000","last_active":"2024-04-02T16:12:14.738979+0000","last_peered":"2024-04-02T16:12:14.738979+0000","last_clean":"2024-04-02T16:12:14.738979+0000","last_became_active":"2024-04-02T16:12:14.732992+0000","last_became_peered":"2024-04-02T16:12:14.732992+0000","last_unstale":"2024-04-02T16:12:14.738979+0000","last_undegraded":"2024-04-02T16:12:14.738979+0000","last_fullsized":"2024-04-02T16:12:14.738979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712203+0000","last_change":"2024-04-02T16:12:14.712203+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712203+0000","last_undegraded":"2024-04-02T16:12:14.712203+0000","last_fullsized":"2024-04-02T16:12:14.712203+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737483+0000","last_change":"2024-04-02T16:12:14.735269+0000","last_active":"2024-04-02T16:12:14.737483+0000","last_peered":"2024-04-02T16:12:14.737483+0000","last_clean":"2024-04-02T16:12:14.737483+0000","last_became_active":"2024-04-02T16:12:14.734879+0000","last_became_peered":"2024-04-02T16:12:14.734879+0000","last_unstale":"2024-04-02T16:12:14.737483+0000","last_undegraded":"2024-04-02T16:12:14.737483+0000","last_fullsized":"2024-04-02T16:12:14.737483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703492+0000","last_change":"2024-04-02T16:12:14.703492+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703492+0000","last_undegraded":"2024-04-02T16:12:14.703492+0000","last_fullsized":"2024-04-02T16:12:14.703492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712365+0000","last_change":"2024-04-02T16:12:14.711385+0000","last_active":"2024-04-02T16:12:14.712365+0000","last_peered":"2024-04-02T16:12:14.712365+0000","last_clean":"2024-04-02T16:12:14.712365+0000","last_became_active":"2024-04-02T16:12:14.711031+0000","last_became_peered":"2024-04-02T16:12:14.711031+0000","last_unstale":"2024-04-02T16:12:14.712365+0000","last_undegraded":"2024-04-02T16:12:14.712365+0000","last_fullsized":"2024-04-02T16:12:14.712365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697881+0000","last_change":"2024-04-02T16:12:14.697881+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697881+0000","last_undegraded":"2024-04-02T16:12:14.697881+0000","last_fullsized":"2024-04-02T16:12:14.697881+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716213+0000","last_change":"2024-04-02T16:12:14.715075+0000","last_active":"2024-04-02T16:12:14.716213+0000","last_peered":"2024-04-02T16:12:14.716213+0000","last_clean":"2024-04-02T16:12:14.716213+0000","last_became_active":"2024-04-02T16:12:14.714602+0000","last_became_peered":"2024-04-02T16:12:14.714602+0000","last_unstale":"2024-04-02T16:12:14.716213+0000","last_undegraded":"2024-04-02T16:12:14.716213+0000","last_fullsized":"2024-04-02T16:12:14.716213+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:19.028049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706098+0000","last_change":"2024-04-02T16:12:14.706098+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706098+0000","last_undegraded":"2024-04-02T16:12:14.706098+0000","last_fullsized":"2024-04-02T16:12:14.706098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722220+0000","last_change":"2024-04-02T16:12:14.719120+0000","last_active":"2024-04-02T16:12:14.722220+0000","last_peered":"2024-04-02T16:12:14.722220+0000","last_clean":"2024-04-02T16:12:14.722220+0000","last_became_active":"2024-04-02T16:12:14.718359+0000","last_became_peered":"2024-04-02T16:12:14.718359+0000","last_unstale":"2024-04-02T16:12:14.722220+0000","last_undegraded":"2024-04-02T16:12:14.722220+0000","last_fullsized":"2024-04-02T16:12:14.722220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:59.701393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707982+0000","last_change":"2024-04-02T16:12:14.707982+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707982+0000","last_undegraded":"2024-04-02T16:12:14.707982+0000","last_fullsized":"2024-04-02T16:12:14.707982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722598+0000","last_change":"2024-04-02T16:12:14.718750+0000","last_active":"2024-04-02T16:12:14.722598+0000","last_peered":"2024-04-02T16:12:14.722598+0000","last_clean":"2024-04-02T16:12:14.722598+0000","last_became_active":"2024-04-02T16:12:14.718131+0000","last_became_peered":"2024-04-02T16:12:14.718131+0000","last_unstale":"2024-04-02T16:12:14.722598+0000","last_undegraded":"2024-04-02T16:12:14.722598+0000","last_fullsized":"2024-04-02T16:12:14.722598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:24.153685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.701650+0000","last_change":"2024-04-02T16:12:14.701650+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.701650+0000","last_undegraded":"2024-04-02T16:12:14.701650+0000","last_fullsized":"2024-04-02T16:12:14.701650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.714009+0000","last_change":"2024-04-02T16:12:14.712471+0000","last_active":"2024-04-02T16:12:14.714009+0000","last_peered":"2024-04-02T16:12:14.714009+0000","last_clean":"2024-04-02T16:12:14.714009+0000","last_became_active":"2024-04-02T16:12:14.711972+0000","last_became_peered":"2024-04-02T16:12:14.711972+0000","last_unstale":"2024-04-02T16:12:14.714009+0000","last_undegraded":"2024-04-02T16:12:14.714009+0000","last_fullsized":"2024-04-02T16:12:14.714009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699448+0000","last_change":"2024-04-02T16:12:14.699448+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699448+0000","last_undegraded":"2024-04-02T16:12:14.699448+0000","last_fullsized":"2024-04-02T16:12:14.699448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729945+0000","last_change":"2024-04-02T16:12:14.725124+0000","last_active":"2024-04-02T16:12:14.729945+0000","last_peered":"2024-04-02T16:12:14.729945+0000","last_clean":"2024-04-02T16:12:14.729945+0000","last_became_active":"2024-04-02T16:12:14.724472+0000","last_became_peered":"2024-04-02T16:12:14.724472+0000","last_unstale":"2024-04-02T16:12:14.729945+0000","last_undegraded":"2024-04-02T16:12:14.729945+0000","last_fullsized":"2024-04-02T16:12:14.729945+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:40.524179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706851+0000","last_change":"2024-04-02T16:12:14.706851+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706851+0000","last_undegraded":"2024-04-02T16:12:14.706851+0000","last_fullsized":"2024-04-02T16:12:14.706851+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737486+0000","last_change":"2024-04-02T16:12:14.735619+0000","last_active":"2024-04-02T16:12:14.737486+0000","last_peered":"2024-04-02T16:12:14.737486+0000","last_clean":"2024-04-02T16:12:14.737486+0000","last_became_active":"2024-04-02T16:12:14.735188+0000","last_became_peered":"2024-04-02T16:12:14.735188+0000","last_unstale":"2024-04-02T16:12:14.737486+0000","last_undegraded":"2024-04-02T16:12:14.737486+0000","last_fullsized":"2024-04-02T16:12:14.737486+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702818+0000","last_change":"2024-04-02T16:12:14.702818+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702818+0000","last_undegraded":"2024-04-02T16:12:14.702818+0000","last_fullsized":"2024-04-02T16:12:14.702818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712128+0000","last_change":"2024-04-02T16:12:14.708255+0000","last_active":"2024-04-02T16:12:14.712128+0000","last_peered":"2024-04-02T16:12:14.712128+0000","last_clean":"2024-04-02T16:12:14.712128+0000","last_became_active":"2024-04-02T16:12:14.707800+0000","last_became_peered":"2024-04-02T16:12:14.707800+0000","last_unstale":"2024-04-02T16:12:14.712128+0000","last_undegraded":"2024-04-02T16:12:14.712128+0000","last_fullsized":"2024-04-02T16:12:14.712128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698603+0000","last_change":"2024-04-02T16:12:14.698603+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698603+0000","last_undegraded":"2024-04-02T16:12:14.698603+0000","last_fullsized":"2024-04-02T16:12:14.698603+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740173+0000","last_change":"2024-04-02T16:12:14.737929+0000","last_active":"2024-04-02T16:12:14.740173+0000","last_peered":"2024-04-02T16:12:14.740173+0000","last_clean":"2024-04-02T16:12:14.740173+0000","last_became_active":"2024-04-02T16:12:14.737178+0000","last_became_peered":"2024-04-02T16:12:14.737178+0000","last_unstale":"2024-04-02T16:12:14.740173+0000","last_undegraded":"2024-04-02T16:12:14.740173+0000","last_fullsized":"2024-04-02T16:12:14.740173+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711895+0000","last_change":"2024-04-02T16:12:14.711895+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711895+0000","last_undegraded":"2024-04-02T16:12:14.711895+0000","last_fullsized":"2024-04-02T16:12:14.711895+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726706+0000","last_change":"2024-04-02T16:12:14.719497+0000","last_active":"2024-04-02T16:12:14.726706+0000","last_peered":"2024-04-02T16:12:14.726706+0000","last_clean":"2024-04-02T16:12:14.726706+0000","last_became_active":"2024-04-02T16:12:14.719166+0000","last_became_peered":"2024-04-02T16:12:14.719166+0000","last_unstale":"2024-04-02T16:12:14.726706+0000","last_undegraded":"2024-04-02T16:12:14.726706+0000","last_fullsized":"2024-04-02T16:12:14.726706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:24.931785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706719+0000","last_change":"2024-04-02T16:12:14.706719+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706719+0000","last_undegraded":"2024-04-02T16:12:14.706719+0000","last_fullsized":"2024-04-02T16:12:14.706719+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733827+0000","last_change":"2024-04-02T16:12:14.726985+0000","last_active":"2024-04-02T16:12:14.733827+0000","last_peered":"2024-04-02T16:12:14.733827+0000","last_clean":"2024-04-02T16:12:14.733827+0000","last_became_active":"2024-04-02T16:12:14.726281+0000","last_became_peered":"2024-04-02T16:12:14.726281+0000","last_unstale":"2024-04-02T16:12:14.733827+0000","last_undegraded":"2024-04-02T16:12:14.733827+0000","last_fullsized":"2024-04-02T16:12:14.733827+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:43.241626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703947+0000","last_change":"2024-04-02T16:12:14.703947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703947+0000","last_undegraded":"2024-04-02T16:12:14.703947+0000","last_fullsized":"2024-04-02T16:12:14.703947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729153+0000","last_change":"2024-04-02T16:12:14.724232+0000","last_active":"2024-04-02T16:12:14.729153+0000","last_peered":"2024-04-02T16:12:14.729153+0000","last_clean":"2024-04-02T16:12:14.729153+0000","last_became_active":"2024-04-02T16:12:14.723889+0000","last_became_peered":"2024-04-02T16:12:14.723889+0000","last_unstale":"2024-04-02T16:12:14.729153+0000","last_undegraded":"2024-04-02T16:12:14.729153+0000","last_fullsized":"2024-04-02T16:12:14.729153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705456+0000","last_change":"2024-04-02T16:12:14.705456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705456+0000","last_undegraded":"2024-04-02T16:12:14.705456+0000","last_fullsized":"2024-04-02T16:12:14.705456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737546+0000","last_change":"2024-04-02T16:12:14.735767+0000","last_active":"2024-04-02T16:12:14.737546+0000","last_peered":"2024-04-02T16:12:14.737546+0000","last_clean":"2024-04-02T16:12:14.737546+0000","last_became_active":"2024-04-02T16:12:14.735499+0000","last_became_peered":"2024-04-02T16:12:14.735499+0000","last_unstale":"2024-04-02T16:12:14.737546+0000","last_undegraded":"2024-04-02T16:12:14.737546+0000","last_fullsized":"2024-04-02T16:12:14.737546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709624+0000","last_change":"2024-04-02T16:12:14.709624+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709624+0000","last_undegraded":"2024-04-02T16:12:14.709624+0000","last_fullsized":"2024-04-02T16:12:14.709624+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716043+0000","last_change":"2024-04-02T16:12:14.714641+0000","last_active":"2024-04-02T16:12:14.716043+0000","last_peered":"2024-04-02T16:12:14.716043+0000","last_clean":"2024-04-02T16:12:14.716043+0000","last_became_active":"2024-04-02T16:12:14.714242+0000","last_became_peered":"2024-04-02T16:12:14.714242+0000","last_unstale":"2024-04-02T16:12:14.716043+0000","last_undegraded":"2024-04-02T16:12:14.716043+0000","last_fullsized":"2024-04-02T16:12:14.716043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703317+0000","last_change":"2024-04-02T16:12:14.703317+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703317+0000","last_undegraded":"2024-04-02T16:12:14.703317+0000","last_fullsized":"2024-04-02T16:12:14.703317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.717058+0000","last_change":"2024-04-02T16:12:14.711749+0000","last_active":"2024-04-02T16:12:14.717058+0000","last_peered":"2024-04-02T16:12:14.717058+0000","last_clean":"2024-04-02T16:12:14.717058+0000","last_became_active":"2024-04-02T16:12:14.711308+0000","last_became_peered":"2024-04-02T16:12:14.711308+0000","last_unstale":"2024-04-02T16:12:14.717058+0000","last_undegraded":"2024-04-02T16:12:14.717058+0000","last_fullsized":"2024-04-02T16:12:14.717058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708089+0000","last_change":"2024-04-02T16:12:14.708089+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708089+0000","last_undegraded":"2024-04-02T16:12:14.708089+0000","last_fullsized":"2024-04-02T16:12:14.708089+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734463+0000","last_change":"2024-04-02T16:12:14.728114+0000","last_active":"2024-04-02T16:12:14.734463+0000","last_peered":"2024-04-02T16:12:14.734463+0000","last_clean":"2024-04-02T16:12:14.734463+0000","last_became_active":"2024-04-02T16:12:14.727429+0000","last_became_peered":"2024-04-02T16:12:14.727429+0000","last_unstale":"2024-04-02T16:12:14.734463+0000","last_undegraded":"2024-04-02T16:12:14.734463+0000","last_fullsized":"2024-04-02T16:12:14.734463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:05.668055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709940+0000","last_change":"2024-04-02T16:12:14.709940+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709940+0000","last_undegraded":"2024-04-02T16:12:14.709940+0000","last_fullsized":"2024-04-02T16:12:14.709940+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722851+0000","last_change":"2024-04-02T16:12:14.716172+0000","last_active":"2024-04-02T16:12:14.722851+0000","last_peered":"2024-04-02T16:12:14.722851+0000","last_clean":"2024-04-02T16:12:14.722851+0000","last_became_active":"2024-04-02T16:12:14.714967+0000","last_became_peered":"2024-04-02T16:12:14.714967+0000","last_unstale":"2024-04-02T16:12:14.722851+0000","last_undegraded":"2024-04-02T16:12:14.722851+0000","last_fullsized":"2024-04-02T16:12:14.722851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:19.185664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697302+0000","last_change":"2024-04-02T16:12:14.697302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697302+0000","last_undegraded":"2024-04-02T16:12:14.697302+0000","last_fullsized":"2024-04-02T16:12:14.697302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722096+0000","last_change":"2024-04-02T16:12:14.718370+0000","last_active":"2024-04-02T16:12:14.722096+0000","last_peered":"2024-04-02T16:12:14.722096+0000","last_clean":"2024-04-02T16:12:14.722096+0000","last_became_active":"2024-04-02T16:12:14.717749+0000","last_became_peered":"2024-04-02T16:12:14.717749+0000","last_unstale":"2024-04-02T16:12:14.722096+0000","last_undegraded":"2024-04-02T16:12:14.722096+0000","last_fullsized":"2024-04-02T16:12:14.722096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:19:34.428530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697456+0000","last_change":"2024-04-02T16:12:14.697456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697456+0000","last_undegraded":"2024-04-02T16:12:14.697456+0000","last_fullsized":"2024-04-02T16:12:14.697456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718780+0000","last_change":"2024-04-02T16:12:14.717818+0000","last_active":"2024-04-02T16:12:14.718780+0000","last_peered":"2024-04-02T16:12:14.718780+0000","last_clean":"2024-04-02T16:12:14.718780+0000","last_became_active":"2024-04-02T16:12:14.717461+0000","last_became_peered":"2024-04-02T16:12:14.717461+0000","last_unstale":"2024-04-02T16:12:14.718780+0000","last_undegraded":"2024-04-02T16:12:14.718780+0000","last_fullsized":"2024-04-02T16:12:14.718780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697306+0000","last_change":"2024-04-02T16:12:14.697306+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697306+0000","last_undegraded":"2024-04-02T16:12:14.697306+0000","last_fullsized":"2024-04-02T16:12:14.697306+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734053+0000","last_change":"2024-04-02T16:12:14.729387+0000","last_active":"2024-04-02T16:12:14.734053+0000","last_peered":"2024-04-02T16:12:14.734053+0000","last_clean":"2024-04-02T16:12:14.734053+0000","last_became_active":"2024-04-02T16:12:14.728744+0000","last_became_peered":"2024-04-02T16:12:14.728744+0000","last_unstale":"2024-04-02T16:12:14.734053+0000","last_undegraded":"2024-04-02T16:12:14.734053+0000","last_fullsized":"2024-04-02T16:12:14.734053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697542+0000","last_change":"2024-04-02T16:12:14.697542+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697542+0000","last_undegraded":"2024-04-02T16:12:14.697542+0000","last_fullsized":"2024-04-02T16:12:14.697542+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733963+0000","last_change":"2024-04-02T16:12:14.728831+0000","last_active":"2024-04-02T16:12:14.733963+0000","last_peered":"2024-04-02T16:12:14.733963+0000","last_clean":"2024-04-02T16:12:14.733963+0000","last_became_active":"2024-04-02T16:12:14.727741+0000","last_became_peered":"2024-04-02T16:12:14.727741+0000","last_unstale":"2024-04-02T16:12:14.733963+0000","last_undegraded":"2024-04-02T16:12:14.733963+0000","last_fullsized":"2024-04-02T16:12:14.733963+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:51.046626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710015+0000","last_change":"2024-04-02T16:12:14.710015+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710015+0000","last_undegraded":"2024-04-02T16:12:14.710015+0000","last_fullsized":"2024-04-02T16:12:14.710015+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740130+0000","last_change":"2024-04-02T16:12:14.737907+0000","last_active":"2024-04-02T16:12:14.740130+0000","last_peered":"2024-04-02T16:12:14.740130+0000","last_clean":"2024-04-02T16:12:14.740130+0000","last_became_active":"2024-04-02T16:12:14.737140+0000","last_became_peered":"2024-04-02T16:12:14.737140+0000","last_unstale":"2024-04-02T16:12:14.740130+0000","last_undegraded":"2024-04-02T16:12:14.740130+0000","last_fullsized":"2024-04-02T16:12:14.740130+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697201+0000","last_change":"2024-04-02T16:12:14.697201+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697201+0000","last_undegraded":"2024-04-02T16:12:14.697201+0000","last_fullsized":"2024-04-02T16:12:14.697201+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729195+0000","last_change":"2024-04-02T16:12:14.725690+0000","last_active":"2024-04-02T16:12:14.729195+0000","last_peered":"2024-04-02T16:12:14.729195+0000","last_clean":"2024-04-02T16:12:14.729195+0000","last_became_active":"2024-04-02T16:12:14.725425+0000","last_became_peered":"2024-04-02T16:12:14.725425+0000","last_unstale":"2024-04-02T16:12:14.729195+0000","last_undegraded":"2024-04-02T16:12:14.729195+0000","last_fullsized":"2024-04-02T16:12:14.729195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706947+0000","last_change":"2024-04-02T16:12:14.706947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706947+0000","last_undegraded":"2024-04-02T16:12:14.706947+0000","last_fullsized":"2024-04-02T16:12:14.706947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.713765+0000","last_change":"2024-04-02T16:12:14.711273+0000","last_active":"2024-04-02T16:12:14.713765+0000","last_peered":"2024-04-02T16:12:14.713765+0000","last_clean":"2024-04-02T16:12:14.713765+0000","last_became_active":"2024-04-02T16:12:14.710880+0000","last_became_peered":"2024-04-02T16:12:14.710880+0000","last_unstale":"2024-04-02T16:12:14.713765+0000","last_undegraded":"2024-04-02T16:12:14.713765+0000","last_fullsized":"2024-04-02T16:12:14.713765+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700493+0000","last_change":"2024-04-02T16:12:14.700493+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700493+0000","last_undegraded":"2024-04-02T16:12:14.700493+0000","last_fullsized":"2024-04-02T16:12:14.700493+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718705+0000","last_change":"2024-04-02T16:12:14.716204+0000","last_active":"2024-04-02T16:12:14.718705+0000","last_peered":"2024-04-02T16:12:14.718705+0000","last_clean":"2024-04-02T16:12:14.718705+0000","last_became_active":"2024-04-02T16:12:14.715693+0000","last_became_peered":"2024-04-02T16:12:14.715693+0000","last_unstale":"2024-04-02T16:12:14.718705+0000","last_undegraded":"2024-04-02T16:12:14.718705+0000","last_fullsized":"2024-04-02T16:12:14.718705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711442+0000","last_change":"2024-04-02T16:12:14.711442+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711442+0000","last_undegraded":"2024-04-02T16:12:14.711442+0000","last_fullsized":"2024-04-02T16:12:14.711442+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726471+0000","last_change":"2024-04-02T16:12:14.723590+0000","last_active":"2024-04-02T16:12:14.726471+0000","last_peered":"2024-04-02T16:12:14.726471+0000","last_clean":"2024-04-02T16:12:14.726471+0000","last_became_active":"2024-04-02T16:12:14.723145+0000","last_became_peered":"2024-04-02T16:12:14.723145+0000","last_unstale":"2024-04-02T16:12:14.726471+0000","last_undegraded":"2024-04-02T16:12:14.726471+0000","last_fullsized":"2024-04-02T16:12:14.726471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703745+0000","last_change":"2024-04-02T16:12:14.703745+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703745+0000","last_undegraded":"2024-04-02T16:12:14.703745+0000","last_fullsized":"2024-04-02T16:12:14.703745+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729200+0000","last_change":"2024-04-02T16:12:14.725705+0000","last_active":"2024-04-02T16:12:14.729200+0000","last_peered":"2024-04-02T16:12:14.729200+0000","last_clean":"2024-04-02T16:12:14.729200+0000","last_became_active":"2024-04-02T16:12:14.725450+0000","last_became_peered":"2024-04-02T16:12:14.725450+0000","last_unstale":"2024-04-02T16:12:14.729200+0000","last_undegraded":"2024-04-02T16:12:14.729200+0000","last_fullsized":"2024-04-02T16:12:14.729200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702914+0000","last_change":"2024-04-02T16:12:14.702914+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702914+0000","last_undegraded":"2024-04-02T16:12:14.702914+0000","last_fullsized":"2024-04-02T16:12:14.702914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734476+0000","last_change":"2024-04-02T16:12:14.726690+0000","last_active":"2024-04-02T16:12:14.734476+0000","last_peered":"2024-04-02T16:12:14.734476+0000","last_clean":"2024-04-02T16:12:14.734476+0000","last_became_active":"2024-04-02T16:12:14.725905+0000","last_became_peered":"2024-04-02T16:12:14.725905+0000","last_unstale":"2024-04-02T16:12:14.734476+0000","last_undegraded":"2024-04-02T16:12:14.734476+0000","last_fullsized":"2024-04-02T16:12:14.734476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:54.536550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704293+0000","last_change":"2024-04-02T16:12:14.704293+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704293+0000","last_undegraded":"2024-04-02T16:12:14.704293+0000","last_fullsized":"2024-04-02T16:12:14.704293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737552+0000","last_change":"2024-04-02T16:12:14.736388+0000","last_active":"2024-04-02T16:12:14.737552+0000","last_peered":"2024-04-02T16:12:14.737552+0000","last_clean":"2024-04-02T16:12:14.737552+0000","last_became_active":"2024-04-02T16:12:14.736096+0000","last_became_peered":"2024-04-02T16:12:14.736096+0000","last_unstale":"2024-04-02T16:12:14.737552+0000","last_undegraded":"2024-04-02T16:12:14.737552+0000","last_fullsized":"2024-04-02T16:12:14.737552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705554+0000","last_change":"2024-04-02T16:12:14.705554+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705554+0000","last_undegraded":"2024-04-02T16:12:14.705554+0000","last_fullsized":"2024-04-02T16:12:14.705554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737786+0000","last_change":"2024-04-02T16:12:14.735597+0000","last_active":"2024-04-02T16:12:14.737786+0000","last_peered":"2024-04-02T16:12:14.737786+0000","last_clean":"2024-04-02T16:12:14.737786+0000","last_became_active":"2024-04-02T16:12:14.734832+0000","last_became_peered":"2024-04-02T16:12:14.734832+0000","last_unstale":"2024-04-02T16:12:14.737786+0000","last_undegraded":"2024-04-02T16:12:14.737786+0000","last_fullsized":"2024-04-02T16:12:14.737786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.720269+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:14.720269+0000","last_peered":"2024-04-02T16:12:14.720269+0000","last_clean":"2024-04-02T16:12:14.720269+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:14.720269+0000","last_undegraded":"2024-04-02T16:12:14.720269+0000","last_fullsized":"2024-04-02T16:12:14.720269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.713236+0000', 'last_change': '2024-04-02T16:12:14.713236+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.713236+0000', 'last_undegraded': '2024-04-02T16:12:14.713236+0000', 'last_fullsized': '2024-04-02T16:12:14.713236+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703207+0000', 'last_change': '2024-04-02T16:12:14.703207+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703207+0000', 'last_undegraded': '2024-04-02T16:12:14.703207+0000', 'last_fullsized': '2024-04-02T16:12:14.703207+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.710853+0000', 'last_change': '2024-04-02T16:12:14.710853+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.710853+0000', 'last_undegraded': '2024-04-02T16:12:14.710853+0000', 'last_fullsized': '2024-04-02T16:12:14.710853+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.712715+0000', 'last_change': '2024-04-02T16:12:14.712715+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.712715+0000', 'last_undegraded': '2024-04-02T16:12:14.712715+0000', 'last_fullsized': '2024-04-02T16:12:14.712715+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.699530+0000', 'last_change': '2024-04-02T16:12:14.699530+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.699530+0000', 'last_undegraded': '2024-04-02T16:12:14.699530+0000', 'last_fullsized': '2024-04-02T16:12:14.699530+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.698314+0000', 'last_change': '2024-04-02T16:12:14.698314+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.698314+0000', 'last_undegraded': '2024-04-02T16:12:14.698314+0000', 'last_fullsized': '2024-04-02T16:12:14.698314+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.704155+0000', 'last_change': '2024-04-02T16:12:14.704155+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.704155+0000', 'last_undegraded': '2024-04-02T16:12:14.704155+0000', 'last_fullsized': '2024-04-02T16:12:14.704155+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.719220+0000', 'last_change': '2024-04-02T16:12:14.719220+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.719220+0000', 'last_undegraded': '2024-04-02T16:12:14.719220+0000', 'last_fullsized': '2024-04-02T16:12:14.719220+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.699751+0000', 'last_change': '2024-04-02T16:12:14.699751+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.699751+0000', 'last_undegraded': '2024-04-02T16:12:14.699751+0000', 'last_fullsized': '2024-04-02T16:12:14.699751+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-02T16:12:19.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.705979+0000', 'last_change': '2024-04-02T16:12:14.705979+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.705979+0000', 'last_undegraded': '2024-04-02T16:12:14.705979+0000', 'last_fullsized': '2024-04-02T16:12:14.705979+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.708302+0000', 'last_change': '2024-04-02T16:12:14.708302+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.708302+0000', 'last_undegraded': '2024-04-02T16:12:14.708302+0000', 'last_fullsized': '2024-04-02T16:12:14.708302+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.707900+0000', 'last_change': '2024-04-02T16:12:14.707900+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.707900+0000', 'last_undegraded': '2024-04-02T16:12:14.707900+0000', 'last_fullsized': '2024-04-02T16:12:14.707900+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.698803+0000', 'last_change': '2024-04-02T16:12:14.698803+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.698803+0000', 'last_undegraded': '2024-04-02T16:12:14.698803+0000', 'last_fullsized': '2024-04-02T16:12:14.698803+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697784+0000', 'last_change': '2024-04-02T16:12:14.697784+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697784+0000', 'last_undegraded': '2024-04-02T16:12:14.697784+0000', 'last_fullsized': '2024-04-02T16:12:14.697784+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.710747+0000', 'last_change': '2024-04-02T16:12:14.710747+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.710747+0000', 'last_undegraded': '2024-04-02T16:12:14.710747+0000', 'last_fullsized': '2024-04-02T16:12:14.710747+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.704126+0000', 'last_change': '2024-04-02T16:12:14.704126+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.704126+0000', 'last_undegraded': '2024-04-02T16:12:14.704126+0000', 'last_fullsized': '2024-04-02T16:12:14.704126+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706721+0000', 'last_change': '2024-04-02T16:12:14.706721+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706721+0000', 'last_undegraded': '2024-04-02T16:12:14.706721+0000', 'last_fullsized': '2024-04-02T16:12:14.706721+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.702833+0000', 'last_change': '2024-04-02T16:12:14.702833+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.702833+0000', 'last_undegraded': '2024-04-02T16:12:14.702833+0000', 'last_fullsized': '2024-04-02T16:12:14.702833+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.702324+0000', 'last_change': '2024-04-02T16:12:14.702324+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.702324+0000', 'last_undegraded': '2024-04-02T16:12:14.702324+0000', 'last_fullsized': '2024-04-02T16:12:14.702324+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697853+0000', 'last_change': '2024-04-02T16:12:14.697853+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697853+0000', 'last_undegraded': '2024-04-02T16:12:14.697853+0000', 'last_fullsized': '2024-04-02T16:12:14.697853+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-02T16:12:19.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703930+0000', 'last_change': '2024-04-02T16:12:14.703930+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703930+0000', 'last_undegraded': '2024-04-02T16:12:14.703930+0000', 'last_fullsized': '2024-04-02T16:12:14.703930+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.700364+0000', 'last_change': '2024-04-02T16:12:14.700364+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.700364+0000', 'last_undegraded': '2024-04-02T16:12:14.700364+0000', 'last_fullsized': '2024-04-02T16:12:14.700364+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.699911+0000', 'last_change': '2024-04-02T16:12:14.699911+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.699911+0000', 'last_undegraded': '2024-04-02T16:12:14.699911+0000', 'last_fullsized': '2024-04-02T16:12:14.699911+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.705612+0000', 'last_change': '2024-04-02T16:12:14.705612+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.705612+0000', 'last_undegraded': '2024-04-02T16:12:14.705612+0000', 'last_fullsized': '2024-04-02T16:12:14.705612+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.700630+0000', 'last_change': '2024-04-02T16:12:14.700630+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.700630+0000', 'last_undegraded': '2024-04-02T16:12:14.700630+0000', 'last_fullsized': '2024-04-02T16:12:14.700630+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.698892+0000', 'last_change': '2024-04-02T16:12:14.698892+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.698892+0000', 'last_undegraded': '2024-04-02T16:12:14.698892+0000', 'last_fullsized': '2024-04-02T16:12:14.698892+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-02T16:12:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.698461+0000', 'last_change': '2024-04-02T16:12:14.698461+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.698461+0000', 'last_undegraded': '2024-04-02T16:12:14.698461+0000', 'last_fullsized': '2024-04-02T16:12:14.698461+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706731+0000', 'last_change': '2024-04-02T16:12:14.706731+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706731+0000', 'last_undegraded': '2024-04-02T16:12:14.706731+0000', 'last_fullsized': '2024-04-02T16:12:14.706731+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.702801+0000', 'last_change': '2024-04-02T16:12:14.702801+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.702801+0000', 'last_undegraded': '2024-04-02T16:12:14.702801+0000', 'last_fullsized': '2024-04-02T16:12:14.702801+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.707213+0000', 'last_change': '2024-04-02T16:12:14.707213+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.707213+0000', 'last_undegraded': '2024-04-02T16:12:14.707213+0000', 'last_fullsized': '2024-04-02T16:12:14.707213+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.710836+0000', 'last_change': '2024-04-02T16:12:14.710836+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.710836+0000', 'last_undegraded': '2024-04-02T16:12:14.710836+0000', 'last_fullsized': '2024-04-02T16:12:14.710836+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.718207+0000', 'last_change': '2024-04-02T16:12:14.718207+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.718207+0000', 'last_undegraded': '2024-04-02T16:12:14.718207+0000', 'last_fullsized': '2024-04-02T16:12:14.718207+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.699274+0000', 'last_change': '2024-04-02T16:12:14.699274+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.699274+0000', 'last_undegraded': '2024-04-02T16:12:14.699274+0000', 'last_fullsized': '2024-04-02T16:12:14.699274+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.712203+0000', 'last_change': '2024-04-02T16:12:14.712203+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.712203+0000', 'last_undegraded': '2024-04-02T16:12:14.712203+0000', 'last_fullsized': '2024-04-02T16:12:14.712203+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703492+0000', 'last_change': '2024-04-02T16:12:14.703492+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703492+0000', 'last_undegraded': '2024-04-02T16:12:14.703492+0000', 'last_fullsized': '2024-04-02T16:12:14.703492+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697881+0000', 'last_change': '2024-04-02T16:12:14.697881+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697881+0000', 'last_undegraded': '2024-04-02T16:12:14.697881+0000', 'last_fullsized': '2024-04-02T16:12:14.697881+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706098+0000', 'last_change': '2024-04-02T16:12:14.706098+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706098+0000', 'last_undegraded': '2024-04-02T16:12:14.706098+0000', 'last_fullsized': '2024-04-02T16:12:14.706098+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.707982+0000', 'last_change': '2024-04-02T16:12:14.707982+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.707982+0000', 'last_undegraded': '2024-04-02T16:12:14.707982+0000', 'last_fullsized': '2024-04-02T16:12:14.707982+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.701650+0000', 'last_change': '2024-04-02T16:12:14.701650+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.701650+0000', 'last_undegraded': '2024-04-02T16:12:14.701650+0000', 'last_fullsized': '2024-04-02T16:12:14.701650+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.699448+0000', 'last_change': '2024-04-02T16:12:14.699448+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.699448+0000', 'last_undegraded': '2024-04-02T16:12:14.699448+0000', 'last_fullsized': '2024-04-02T16:12:14.699448+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706851+0000', 'last_change': '2024-04-02T16:12:14.706851+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706851+0000', 'last_undegraded': '2024-04-02T16:12:14.706851+0000', 'last_fullsized': '2024-04-02T16:12:14.706851+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.702818+0000', 'last_change': '2024-04-02T16:12:14.702818+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.702818+0000', 'last_undegraded': '2024-04-02T16:12:14.702818+0000', 'last_fullsized': '2024-04-02T16:12:14.702818+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-02T16:12:19.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.698603+0000', 'last_change': '2024-04-02T16:12:14.698603+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.698603+0000', 'last_undegraded': '2024-04-02T16:12:14.698603+0000', 'last_fullsized': '2024-04-02T16:12:14.698603+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.711895+0000', 'last_change': '2024-04-02T16:12:14.711895+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.711895+0000', 'last_undegraded': '2024-04-02T16:12:14.711895+0000', 'last_fullsized': '2024-04-02T16:12:14.711895+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706719+0000', 'last_change': '2024-04-02T16:12:14.706719+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706719+0000', 'last_undegraded': '2024-04-02T16:12:14.706719+0000', 'last_fullsized': '2024-04-02T16:12:14.706719+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703947+0000', 'last_change': '2024-04-02T16:12:14.703947+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703947+0000', 'last_undegraded': '2024-04-02T16:12:14.703947+0000', 'last_fullsized': '2024-04-02T16:12:14.703947+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.705456+0000', 'last_change': '2024-04-02T16:12:14.705456+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.705456+0000', 'last_undegraded': '2024-04-02T16:12:14.705456+0000', 'last_fullsized': '2024-04-02T16:12:14.705456+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-02T16:12:19.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.709624+0000', 'last_change': '2024-04-02T16:12:14.709624+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.709624+0000', 'last_undegraded': '2024-04-02T16:12:14.709624+0000', 'last_fullsized': '2024-04-02T16:12:14.709624+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703317+0000', 'last_change': '2024-04-02T16:12:14.703317+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703317+0000', 'last_undegraded': '2024-04-02T16:12:14.703317+0000', 'last_fullsized': '2024-04-02T16:12:14.703317+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.708089+0000', 'last_change': '2024-04-02T16:12:14.708089+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.708089+0000', 'last_undegraded': '2024-04-02T16:12:14.708089+0000', 'last_fullsized': '2024-04-02T16:12:14.708089+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.709940+0000', 'last_change': '2024-04-02T16:12:14.709940+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.709940+0000', 'last_undegraded': '2024-04-02T16:12:14.709940+0000', 'last_fullsized': '2024-04-02T16:12:14.709940+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697302+0000', 'last_change': '2024-04-02T16:12:14.697302+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697302+0000', 'last_undegraded': '2024-04-02T16:12:14.697302+0000', 'last_fullsized': '2024-04-02T16:12:14.697302+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697456+0000', 'last_change': '2024-04-02T16:12:14.697456+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697456+0000', 'last_undegraded': '2024-04-02T16:12:14.697456+0000', 'last_fullsized': '2024-04-02T16:12:14.697456+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697306+0000', 'last_change': '2024-04-02T16:12:14.697306+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697306+0000', 'last_undegraded': '2024-04-02T16:12:14.697306+0000', 'last_fullsized': '2024-04-02T16:12:14.697306+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697542+0000', 'last_change': '2024-04-02T16:12:14.697542+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697542+0000', 'last_undegraded': '2024-04-02T16:12:14.697542+0000', 'last_fullsized': '2024-04-02T16:12:14.697542+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.710015+0000', 'last_change': '2024-04-02T16:12:14.710015+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.710015+0000', 'last_undegraded': '2024-04-02T16:12:14.710015+0000', 'last_fullsized': '2024-04-02T16:12:14.710015+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.697201+0000', 'last_change': '2024-04-02T16:12:14.697201+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.697201+0000', 'last_undegraded': '2024-04-02T16:12:14.697201+0000', 'last_fullsized': '2024-04-02T16:12:14.697201+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.706947+0000', 'last_change': '2024-04-02T16:12:14.706947+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.706947+0000', 'last_undegraded': '2024-04-02T16:12:14.706947+0000', 'last_fullsized': '2024-04-02T16:12:14.706947+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-02T16:12:19.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.700493+0000', 'last_change': '2024-04-02T16:12:14.700493+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.700493+0000', 'last_undegraded': '2024-04-02T16:12:14.700493+0000', 'last_fullsized': '2024-04-02T16:12:14.700493+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.711442+0000', 'last_change': '2024-04-02T16:12:14.711442+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.711442+0000', 'last_undegraded': '2024-04-02T16:12:14.711442+0000', 'last_fullsized': '2024-04-02T16:12:14.711442+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.703745+0000', 'last_change': '2024-04-02T16:12:14.703745+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.703745+0000', 'last_undegraded': '2024-04-02T16:12:14.703745+0000', 'last_fullsized': '2024-04-02T16:12:14.703745+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.702914+0000', 'last_change': '2024-04-02T16:12:14.702914+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.702914+0000', 'last_undegraded': '2024-04-02T16:12:14.702914+0000', 'last_fullsized': '2024-04-02T16:12:14.702914+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.704293+0000', 'last_change': '2024-04-02T16:12:14.704293+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.704293+0000', 'last_undegraded': '2024-04-02T16:12:14.704293+0000', 'last_fullsized': '2024-04-02T16:12:14.704293+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-02T16:12:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 27, 'state': 'creating+peering', 'last_fresh': '2024-04-02T16:12:14.705554+0000', 'last_change': '2024-04-02T16:12:14.705554+0000', 'last_active': '2024-04-02T16:12:14.679647+0000', 'last_peered': '2024-04-02T16:12:14.679647+0000', 'last_clean': '2024-04-02T16:12:14.679647+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-02T16:12:14.705554+0000', 'last_undegraded': '2024-04-02T16:12:14.705554+0000', 'last_fullsized': '2024-04-02T16:12:14.705554+0000', 'mapping_epoch': 27, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 27, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'last_clean_scrub_stamp': '2024-04-02T16:12:14.679647+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-02T16:12:19.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-02T16:12:19.650 INFO:tasks.ceph.mds.a.smithi040.stderr:2024-04-02T16:12:19.648+0000 7f5e585c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:12:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 Processor -- start 2024-04-02T16:12:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 -- start start 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a34150110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340b51c0 con 0x7f3a34147d20 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340b3d40 con 0x7f3a340b3960 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a39d4c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3a340b0e20 con 0x7f3a3414dd20 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a32ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3a34147d20 0x7f3a34148180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55930/0 (socket says 172.21.15.40:55930) 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a32ffd640 1 -- 172.21.15.40:0/3972837363 learned_addr learned my addr 172.21.15.40:0/3972837363 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a33fff640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a34150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744979839 0 0) 0x7f3a340b51c0 con 0x7f3a34147d20 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a1c003530 con 0x7f3a34147d20 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 336151393 0 0) 0x7f3a1c003530 con 0x7f3a34147d20 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 msgr2=0x7f3a34150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a34150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 >> v1:172.21.15.204:6789/0 conn(0x7f3a340b3960 legacy=0x7f3a34072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340af930 con 0x7f3a34147d20 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f3a280016a0 con 0x7f3a34147d20 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 >> v1:172.21.15.40:6789/0 conn(0x7f3a34147d20 legacy=0x7f3a34148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0039f0 0x7f3a1c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a1c0052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340b0e20 con 0x7f3a1c004810 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c00bb90 con 0x7f3a1c0039f0 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.668+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1c00bd90 con 0x7f3a3414dd20 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 -- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 msgr2=0x7f3a1c0052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a337fe640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0039f0 0x7f3a1c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a33fff640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a1c0052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a3414dd20 0x7f3a1c0052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 -- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0039f0 msgr2=0x7f3a1c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0039f0 0x7f3a1c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 -- 172.21.15.40:0/3972837363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c003530 con 0x7f3a1c004810 2024-04-02T16:12:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a337fe640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0039f0 0x7f3a1c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a1c004c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3a340b5390 tx=0x7f3a2801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc091437002e980 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28007d20 con 0x7f3a1c004810 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a280a9020 con 0x7f3a1c004810 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a327fc640 1 -- 172.21.15.40:0/3972837363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28005b20 con 0x7f3a1c004810 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 -- 172.21.15.40:0/3972837363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 msgr2=0x7f3a1c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/3972837363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a1c004c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3a340b5390 tx=0x7f3a2801cc00 comp rx=0 tx=0).stop 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a33fff640 1 -- 172.21.15.40:0/3972837363 reap_dead start 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 -- 172.21.15.40:0/3972837363 shutdown_connections 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 -- 172.21.15.40:0/3972837363 >> 172.21.15.40:0/3972837363 conn(0x7f3a340b5800 msgr2=0x7f3a340b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 -- 172.21.15.40:0/3972837363 shutdown_connections 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.672+0000 7f3a39d4c640 1 -- 172.21.15.40:0/3972837363 wait complete. 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 Processor -- start 2024-04-02T16:12:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- start start 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a340b8340 0x7f3a3413cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a341e01b0 0x7f3a341442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340af6f0 con 0x7f3a1c004810 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3406dcb0 con 0x7f3a340b8340 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3a34150920 con 0x7f3a341e01b0 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59294/0 (socket says 172.21.15.40:59294) 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 -- 172.21.15.40:0/1306520525 learned_addr learned my addr 172.21.15.40:0/1306520525 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a340b8340 0x7f3a3413cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a33fff640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a341e01b0 0x7f3a341442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 -- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a341e01b0 msgr2=0x7f3a341442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a341e01b0 0x7f3a341442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 -- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a340b8340 msgr2=0x7f3a3413cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a340b8340 0x7f3a3413cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 -- 172.21.15.40:0/1306520525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3413e150 con 0x7f3a1c004810 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a337fe640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d570 tx=0x7f3a2400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24005480 con 0x7f3a1c004810 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24017070 con 0x7f3a1c004810 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a341e0630 con 0x7f3a1c004810 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003840 con 0x7f3a1c004810 2024-04-02T16:12:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a34145970 con 0x7f3a1c004810 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a340685f0 con 0x7f3a1c004810 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3a240039e0 con 0x7f3a1c004810 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 0x7f3a2006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3a24086690 con 0x7f3a1c004810 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.676+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 0x7f3a2006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.680+0000 7f3a32ffd640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 0x7f3a2006a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a280aa000 tx=0x7f3a34145450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.680+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3a341e0630 con 0x7f3a1c004810 2024-04-02T16:12:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.904+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3a340b0860 con 0x7f3a1c004810 2024-04-02T16:12:19.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.904+0000 7f3a1bfff640 1 -- 172.21.15.40:0/1306520525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2388 (secure 0 0 0) 0x7f3a2400f2f0 con 0x7f3a1c004810 2024-04-02T16:12:19.906 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:19.906 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":72,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":8,"num_up_osds":8,"osd_up_since":1712074279,"num_in_osds":8,"osd_in_since":1712074269,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224866304,"bytes_avail":772869246976,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4989}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-02T16:12:13.862237+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-02T16:12:19.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 msgr2=0x7f3a2006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 0x7f3a2006a840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a280aa000 tx=0x7f3a34145450 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 msgr2=0x7f3a341e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d570 tx=0x7f3a2400da30 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 shutdown_connections 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a341e01b0 0x7f3a341442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3a20068380 0x7f3a2006a840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a340b8340 0x7f3a3413cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 --2- 172.21.15.40:0/1306520525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a1c004810 0x7f3a341e4200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 >> 172.21.15.40:0/1306520525 conn(0x7f3a340b5800 msgr2=0x7f3a341382d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 shutdown_connections 2024-04-02T16:12:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:19.908+0000 7f3a39d4c640 1 -- 172.21.15.40:0/1306520525 wait complete. 2024-04-02T16:12:19.925 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-02T16:12:19.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 Processor -- start 2024-04-02T16:12:20.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 -- start start 2024-04-02T16:12:20.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2240b5a00 0x7fc22414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2240b4c20 con 0x7fc22414b240 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc2240b44b0 con 0x7fc2240b5a00 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc228f93640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc2240b6f10 con 0x7fc2240ba020 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2240b5a00 0x7fc22414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2240b5a00 0x7fc22414ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59298/0 (socket says 172.21.15.40:59298) 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 -- 172.21.15.40:0/2714395422 learned_addr learned my addr 172.21.15.40:0/2714395422 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc222d9d640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 -- 172.21.15.40:0/2714395422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 msgr2=0x7fc224152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 -- 172.21.15.40:0/2714395422 >> v1:172.21.15.204:6789/0 conn(0x7fc22414b240 legacy=0x7fc224141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.048+0000 7fc22259c640 1 -- 172.21.15.40:0/2714395422 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2240ba540 con 0x7fc2240b5a00 2024-04-02T16:12:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414ad00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc2140098a0 tx=0x7fc21401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5df41cbd156fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc221d9b640 1 -- 172.21.15.40:0/2714395422 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140ab5f0 con 0x7fc2240b5a00 2024-04-02T16:12:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc221d9b640 1 -- 172.21.15.40:0/2714395422 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2140b0070 con 0x7fc2240b5a00 2024-04-02T16:12:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc221d9b640 1 -- 172.21.15.40:0/2714395422 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140ab9b0 con 0x7fc2240b5a00 2024-04-02T16:12:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- 172.21.15.40:0/2714395422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 msgr2=0x7fc22414ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414ad00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc2140098a0 tx=0x7fc21401c3f0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- 172.21.15.40:0/2714395422 shutdown_connections 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- 172.21.15.40:0/2714395422 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414ad00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- 172.21.15.40:0/2714395422 >> 172.21.15.40:0/2714395422 conn(0x7fc2240730b0 msgr2=0x7fc2240734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- 172.21.15.40:0/2714395422 shutdown_connections 2024-04-02T16:12:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- 172.21.15.40:0/2714395422 wait complete. 2024-04-02T16:12:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 Processor -- start 2024-04-02T16:12:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- start start 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22414b240 0x7fc2241e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2240b6940 con 0x7fc2240b5a00 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc2240b67c0 con 0x7fc22414b240 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc228f93640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc224154df0 con 0x7fc2240ba020 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59312/0 (socket says 172.21.15.40:59312) 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 -- 172.21.15.40:0/2644992689 learned_addr learned my addr 172.21.15.40:0/2644992689 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 -- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 msgr2=0x7fc224142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 -- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22414b240 msgr2=0x7fc2241e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22414b240 0x7fc2241e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 -- 172.21.15.40:0/2644992689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22414e1d0 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.052+0000 7fc22259c640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2140ad040 tx=0x7fc21401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140be5f0 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2140be790 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22414eec0 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2140beb80 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22414def0 con 0x7fc2240b5a00 2024-04-02T16:12:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2240685f0 con 0x7fc2240b5a00 2024-04-02T16:12:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fc2140b43f0 con 0x7fc2240b5a00 2024-04-02T16:12:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 0x7fc20406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fc214121440 con 0x7fc2240b5a00 2024-04-02T16:12:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21bfff640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 0x7fc20406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.056+0000 7fc21bfff640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 0x7fc20406a840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000f70 tx=0x7fc20c0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.060+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fc22414e1d0 con 0x7fc2240b5a00 2024-04-02T16:12:20.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.216+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc22414f1a0 con 0x7fc204068380 2024-04-02T16:12:20.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.232+0000 7fc21affd640 1 -- 172.21.15.40:0/2644992689 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246633 (secure 0 0 0) 0x7fc22414f1a0 con 0x7fc204068380 2024-04-02T16:12:20.236 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 msgr2=0x7fc20406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 0x7fc20406a840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc20c000f70 tx=0x7fc20c0079c0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 msgr2=0x7fc22414c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2140ad040 tx=0x7fc21401ac30 comp rx=0 tx=0).stop 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 shutdown_connections 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2240ba020 0x7fc224142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fc204068380 0x7fc20406a840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22414b240 0x7fc2241e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 --2- 172.21.15.40:0/2644992689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2240b5a00 0x7fc22414c570 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 >> 172.21.15.40:0/2644992689 conn(0x7fc2240730b0 msgr2=0x7fc2240baf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 shutdown_connections 2024-04-02T16:12:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:20.236+0000 7fc228f93640 1 -- 172.21.15.40:0/2644992689 wait complete. 2024-04-02T16:12:20.253 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-02T16:12:19.864122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219596,"kb_used_data":3436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755124,"statfs":{"total":773094113280,"available":772869246976,"internally_reserved":0,"allocated":3518464,"data_stored":2036897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001958"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.713236+0000","last_change":"2024-04-02T16:12:14.713236+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.713236+0000","last_undegraded":"2024-04-02T16:12:14.713236+0000","last_fullsized":"2024-04-02T16:12:14.713236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703207+0000","last_change":"2024-04-02T16:12:14.703207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703207+0000","last_undegraded":"2024-04-02T16:12:14.703207+0000","last_fullsized":"2024-04-02T16:12:14.703207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710853+0000","last_change":"2024-04-02T16:12:14.710853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710853+0000","last_undegraded":"2024-04-02T16:12:14.710853+0000","last_fullsized":"2024-04-02T16:12:14.710853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712715+0000","last_change":"2024-04-02T16:12:14.712715+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712715+0000","last_undegraded":"2024-04-02T16:12:14.712715+0000","last_fullsized":"2024-04-02T16:12:14.712715+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699530+0000","last_change":"2024-04-02T16:12:14.699530+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699530+0000","last_undegraded":"2024-04-02T16:12:14.699530+0000","last_fullsized":"2024-04-02T16:12:14.699530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698314+0000","last_change":"2024-04-02T16:12:14.698314+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698314+0000","last_undegraded":"2024-04-02T16:12:14.698314+0000","last_fullsized":"2024-04-02T16:12:14.698314+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704155+0000","last_change":"2024-04-02T16:12:14.704155+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704155+0000","last_undegraded":"2024-04-02T16:12:14.704155+0000","last_fullsized":"2024-04-02T16:12:14.704155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.719220+0000","last_change":"2024-04-02T16:12:14.719220+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.719220+0000","last_undegraded":"2024-04-02T16:12:14.719220+0000","last_fullsized":"2024-04-02T16:12:14.719220+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699751+0000","last_change":"2024-04-02T16:12:14.699751+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699751+0000","last_undegraded":"2024-04-02T16:12:14.699751+0000","last_fullsized":"2024-04-02T16:12:14.699751+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705979+0000","last_change":"2024-04-02T16:12:14.705979+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705979+0000","last_undegraded":"2024-04-02T16:12:14.705979+0000","last_fullsized":"2024-04-02T16:12:14.705979+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708302+0000","last_change":"2024-04-02T16:12:14.708302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708302+0000","last_undegraded":"2024-04-02T16:12:14.708302+0000","last_fullsized":"2024-04-02T16:12:14.708302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707900+0000","last_change":"2024-04-02T16:12:14.707900+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707900+0000","last_undegraded":"2024-04-02T16:12:14.707900+0000","last_fullsized":"2024-04-02T16:12:14.707900+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698803+0000","last_change":"2024-04-02T16:12:14.698803+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698803+0000","last_undegraded":"2024-04-02T16:12:14.698803+0000","last_fullsized":"2024-04-02T16:12:14.698803+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697784+0000","last_change":"2024-04-02T16:12:14.697784+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697784+0000","last_undegraded":"2024-04-02T16:12:14.697784+0000","last_fullsized":"2024-04-02T16:12:14.697784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710747+0000","last_change":"2024-04-02T16:12:14.710747+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710747+0000","last_undegraded":"2024-04-02T16:12:14.710747+0000","last_fullsized":"2024-04-02T16:12:14.710747+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704126+0000","last_change":"2024-04-02T16:12:14.704126+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704126+0000","last_undegraded":"2024-04-02T16:12:14.704126+0000","last_fullsized":"2024-04-02T16:12:14.704126+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706721+0000","last_change":"2024-04-02T16:12:14.706721+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706721+0000","last_undegraded":"2024-04-02T16:12:14.706721+0000","last_fullsized":"2024-04-02T16:12:14.706721+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702833+0000","last_change":"2024-04-02T16:12:14.702833+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702833+0000","last_undegraded":"2024-04-02T16:12:14.702833+0000","last_fullsized":"2024-04-02T16:12:14.702833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702324+0000","last_change":"2024-04-02T16:12:14.702324+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702324+0000","last_undegraded":"2024-04-02T16:12:14.702324+0000","last_fullsized":"2024-04-02T16:12:14.702324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697853+0000","last_change":"2024-04-02T16:12:14.697853+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697853+0000","last_undegraded":"2024-04-02T16:12:14.697853+0000","last_fullsized":"2024-04-02T16:12:14.697853+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703930+0000","last_change":"2024-04-02T16:12:14.703930+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703930+0000","last_undegraded":"2024-04-02T16:12:14.703930+0000","last_fullsized":"2024-04-02T16:12:14.703930+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700364+0000","last_change":"2024-04-02T16:12:14.700364+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700364+0000","last_undegraded":"2024-04-02T16:12:14.700364+0000","last_fullsized":"2024-04-02T16:12:14.700364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699911+0000","last_change":"2024-04-02T16:12:14.699911+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699911+0000","last_undegraded":"2024-04-02T16:12:14.699911+0000","last_fullsized":"2024-04-02T16:12:14.699911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705612+0000","last_change":"2024-04-02T16:12:14.705612+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705612+0000","last_undegraded":"2024-04-02T16:12:14.705612+0000","last_fullsized":"2024-04-02T16:12:14.705612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700630+0000","last_change":"2024-04-02T16:12:14.700630+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700630+0000","last_undegraded":"2024-04-02T16:12:14.700630+0000","last_fullsized":"2024-04-02T16:12:14.700630+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698892+0000","last_change":"2024-04-02T16:12:14.698892+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698892+0000","last_undegraded":"2024-04-02T16:12:14.698892+0000","last_fullsized":"2024-04-02T16:12:14.698892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698461+0000","last_change":"2024-04-02T16:12:14.698461+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698461+0000","last_undegraded":"2024-04-02T16:12:14.698461+0000","last_fullsized":"2024-04-02T16:12:14.698461+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706731+0000","last_change":"2024-04-02T16:12:14.706731+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706731+0000","last_undegraded":"2024-04-02T16:12:14.706731+0000","last_fullsized":"2024-04-02T16:12:14.706731+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702801+0000","last_change":"2024-04-02T16:12:14.702801+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702801+0000","last_undegraded":"2024-04-02T16:12:14.702801+0000","last_fullsized":"2024-04-02T16:12:14.702801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707213+0000","last_change":"2024-04-02T16:12:14.707213+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707213+0000","last_undegraded":"2024-04-02T16:12:14.707213+0000","last_fullsized":"2024-04-02T16:12:14.707213+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710836+0000","last_change":"2024-04-02T16:12:14.710836+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710836+0000","last_undegraded":"2024-04-02T16:12:14.710836+0000","last_fullsized":"2024-04-02T16:12:14.710836+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.718207+0000","last_change":"2024-04-02T16:12:14.718207+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.718207+0000","last_undegraded":"2024-04-02T16:12:14.718207+0000","last_fullsized":"2024-04-02T16:12:14.718207+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699274+0000","last_change":"2024-04-02T16:12:14.699274+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699274+0000","last_undegraded":"2024-04-02T16:12:14.699274+0000","last_fullsized":"2024-04-02T16:12:14.699274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.738979+0000","last_change":"2024-04-02T16:12:14.733795+0000","last_active":"2024-04-02T16:12:14.738979+0000","last_peered":"2024-04-02T16:12:14.738979+0000","last_clean":"2024-04-02T16:12:14.738979+0000","last_became_active":"2024-04-02T16:12:14.732992+0000","last_became_peered":"2024-04-02T16:12:14.732992+0000","last_unstale":"2024-04-02T16:12:14.738979+0000","last_undegraded":"2024-04-02T16:12:14.738979+0000","last_fullsized":"2024-04-02T16:12:14.738979+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.712203+0000","last_change":"2024-04-02T16:12:14.712203+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.712203+0000","last_undegraded":"2024-04-02T16:12:14.712203+0000","last_fullsized":"2024-04-02T16:12:14.712203+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737483+0000","last_change":"2024-04-02T16:12:14.735269+0000","last_active":"2024-04-02T16:12:14.737483+0000","last_peered":"2024-04-02T16:12:14.737483+0000","last_clean":"2024-04-02T16:12:14.737483+0000","last_became_active":"2024-04-02T16:12:14.734879+0000","last_became_peered":"2024-04-02T16:12:14.734879+0000","last_unstale":"2024-04-02T16:12:14.737483+0000","last_undegraded":"2024-04-02T16:12:14.737483+0000","last_fullsized":"2024-04-02T16:12:14.737483+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703492+0000","last_change":"2024-04-02T16:12:14.703492+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703492+0000","last_undegraded":"2024-04-02T16:12:14.703492+0000","last_fullsized":"2024-04-02T16:12:14.703492+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712365+0000","last_change":"2024-04-02T16:12:14.711385+0000","last_active":"2024-04-02T16:12:14.712365+0000","last_peered":"2024-04-02T16:12:14.712365+0000","last_clean":"2024-04-02T16:12:14.712365+0000","last_became_active":"2024-04-02T16:12:14.711031+0000","last_became_peered":"2024-04-02T16:12:14.711031+0000","last_unstale":"2024-04-02T16:12:14.712365+0000","last_undegraded":"2024-04-02T16:12:14.712365+0000","last_fullsized":"2024-04-02T16:12:14.712365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697881+0000","last_change":"2024-04-02T16:12:14.697881+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697881+0000","last_undegraded":"2024-04-02T16:12:14.697881+0000","last_fullsized":"2024-04-02T16:12:14.697881+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716213+0000","last_change":"2024-04-02T16:12:14.715075+0000","last_active":"2024-04-02T16:12:14.716213+0000","last_peered":"2024-04-02T16:12:14.716213+0000","last_clean":"2024-04-02T16:12:14.716213+0000","last_became_active":"2024-04-02T16:12:14.714602+0000","last_became_peered":"2024-04-02T16:12:14.714602+0000","last_unstale":"2024-04-02T16:12:14.716213+0000","last_undegraded":"2024-04-02T16:12:14.716213+0000","last_fullsized":"2024-04-02T16:12:14.716213+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:09:19.028049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706098+0000","last_change":"2024-04-02T16:12:14.706098+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706098+0000","last_undegraded":"2024-04-02T16:12:14.706098+0000","last_fullsized":"2024-04-02T16:12:14.706098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722220+0000","last_change":"2024-04-02T16:12:14.719120+0000","last_active":"2024-04-02T16:12:14.722220+0000","last_peered":"2024-04-02T16:12:14.722220+0000","last_clean":"2024-04-02T16:12:14.722220+0000","last_became_active":"2024-04-02T16:12:14.718359+0000","last_became_peered":"2024-04-02T16:12:14.718359+0000","last_unstale":"2024-04-02T16:12:14.722220+0000","last_undegraded":"2024-04-02T16:12:14.722220+0000","last_fullsized":"2024-04-02T16:12:14.722220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:59.701393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.707982+0000","last_change":"2024-04-02T16:12:14.707982+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.707982+0000","last_undegraded":"2024-04-02T16:12:14.707982+0000","last_fullsized":"2024-04-02T16:12:14.707982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722598+0000","last_change":"2024-04-02T16:12:14.718750+0000","last_active":"2024-04-02T16:12:14.722598+0000","last_peered":"2024-04-02T16:12:14.722598+0000","last_clean":"2024-04-02T16:12:14.722598+0000","last_became_active":"2024-04-02T16:12:14.718131+0000","last_became_peered":"2024-04-02T16:12:14.718131+0000","last_unstale":"2024-04-02T16:12:14.722598+0000","last_undegraded":"2024-04-02T16:12:14.722598+0000","last_fullsized":"2024-04-02T16:12:14.722598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:21:24.153685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.701650+0000","last_change":"2024-04-02T16:12:14.701650+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.701650+0000","last_undegraded":"2024-04-02T16:12:14.701650+0000","last_fullsized":"2024-04-02T16:12:14.701650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.714009+0000","last_change":"2024-04-02T16:12:14.712471+0000","last_active":"2024-04-02T16:12:14.714009+0000","last_peered":"2024-04-02T16:12:14.714009+0000","last_clean":"2024-04-02T16:12:14.714009+0000","last_became_active":"2024-04-02T16:12:14.711972+0000","last_became_peered":"2024-04-02T16:12:14.711972+0000","last_unstale":"2024-04-02T16:12:14.714009+0000","last_undegraded":"2024-04-02T16:12:14.714009+0000","last_fullsized":"2024-04-02T16:12:14.714009+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.699448+0000","last_change":"2024-04-02T16:12:14.699448+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.699448+0000","last_undegraded":"2024-04-02T16:12:14.699448+0000","last_fullsized":"2024-04-02T16:12:14.699448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729945+0000","last_change":"2024-04-02T16:12:14.725124+0000","last_active":"2024-04-02T16:12:14.729945+0000","last_peered":"2024-04-02T16:12:14.729945+0000","last_clean":"2024-04-02T16:12:14.729945+0000","last_became_active":"2024-04-02T16:12:14.724472+0000","last_became_peered":"2024-04-02T16:12:14.724472+0000","last_unstale":"2024-04-02T16:12:14.729945+0000","last_undegraded":"2024-04-02T16:12:14.729945+0000","last_fullsized":"2024-04-02T16:12:14.729945+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:19:40.524179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706851+0000","last_change":"2024-04-02T16:12:14.706851+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706851+0000","last_undegraded":"2024-04-02T16:12:14.706851+0000","last_fullsized":"2024-04-02T16:12:14.706851+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737486+0000","last_change":"2024-04-02T16:12:14.735619+0000","last_active":"2024-04-02T16:12:14.737486+0000","last_peered":"2024-04-02T16:12:14.737486+0000","last_clean":"2024-04-02T16:12:14.737486+0000","last_became_active":"2024-04-02T16:12:14.735188+0000","last_became_peered":"2024-04-02T16:12:14.735188+0000","last_unstale":"2024-04-02T16:12:14.737486+0000","last_undegraded":"2024-04-02T16:12:14.737486+0000","last_fullsized":"2024-04-02T16:12:14.737486+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702818+0000","last_change":"2024-04-02T16:12:14.702818+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702818+0000","last_undegraded":"2024-04-02T16:12:14.702818+0000","last_fullsized":"2024-04-02T16:12:14.702818+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.712128+0000","last_change":"2024-04-02T16:12:14.708255+0000","last_active":"2024-04-02T16:12:14.712128+0000","last_peered":"2024-04-02T16:12:14.712128+0000","last_clean":"2024-04-02T16:12:14.712128+0000","last_became_active":"2024-04-02T16:12:14.707800+0000","last_became_peered":"2024-04-02T16:12:14.707800+0000","last_unstale":"2024-04-02T16:12:14.712128+0000","last_undegraded":"2024-04-02T16:12:14.712128+0000","last_fullsized":"2024-04-02T16:12:14.712128+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.698603+0000","last_change":"2024-04-02T16:12:14.698603+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.698603+0000","last_undegraded":"2024-04-02T16:12:14.698603+0000","last_fullsized":"2024-04-02T16:12:14.698603+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740173+0000","last_change":"2024-04-02T16:12:14.737929+0000","last_active":"2024-04-02T16:12:14.740173+0000","last_peered":"2024-04-02T16:12:14.740173+0000","last_clean":"2024-04-02T16:12:14.740173+0000","last_became_active":"2024-04-02T16:12:14.737178+0000","last_became_peered":"2024-04-02T16:12:14.737178+0000","last_unstale":"2024-04-02T16:12:14.740173+0000","last_undegraded":"2024-04-02T16:12:14.740173+0000","last_fullsized":"2024-04-02T16:12:14.740173+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711895+0000","last_change":"2024-04-02T16:12:14.711895+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711895+0000","last_undegraded":"2024-04-02T16:12:14.711895+0000","last_fullsized":"2024-04-02T16:12:14.711895+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726706+0000","last_change":"2024-04-02T16:12:14.719497+0000","last_active":"2024-04-02T16:12:14.726706+0000","last_peered":"2024-04-02T16:12:14.726706+0000","last_clean":"2024-04-02T16:12:14.726706+0000","last_became_active":"2024-04-02T16:12:14.719166+0000","last_became_peered":"2024-04-02T16:12:14.719166+0000","last_unstale":"2024-04-02T16:12:14.726706+0000","last_undegraded":"2024-04-02T16:12:14.726706+0000","last_fullsized":"2024-04-02T16:12:14.726706+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:24.931785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706719+0000","last_change":"2024-04-02T16:12:14.706719+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706719+0000","last_undegraded":"2024-04-02T16:12:14.706719+0000","last_fullsized":"2024-04-02T16:12:14.706719+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733827+0000","last_change":"2024-04-02T16:12:14.726985+0000","last_active":"2024-04-02T16:12:14.733827+0000","last_peered":"2024-04-02T16:12:14.733827+0000","last_clean":"2024-04-02T16:12:14.733827+0000","last_became_active":"2024-04-02T16:12:14.726281+0000","last_became_peered":"2024-04-02T16:12:14.726281+0000","last_unstale":"2024-04-02T16:12:14.733827+0000","last_undegraded":"2024-04-02T16:12:14.733827+0000","last_fullsized":"2024-04-02T16:12:14.733827+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:34:43.241626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703947+0000","last_change":"2024-04-02T16:12:14.703947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703947+0000","last_undegraded":"2024-04-02T16:12:14.703947+0000","last_fullsized":"2024-04-02T16:12:14.703947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729153+0000","last_change":"2024-04-02T16:12:14.724232+0000","last_active":"2024-04-02T16:12:14.729153+0000","last_peered":"2024-04-02T16:12:14.729153+0000","last_clean":"2024-04-02T16:12:14.729153+0000","last_became_active":"2024-04-02T16:12:14.723889+0000","last_became_peered":"2024-04-02T16:12:14.723889+0000","last_unstale":"2024-04-02T16:12:14.729153+0000","last_undegraded":"2024-04-02T16:12:14.729153+0000","last_fullsized":"2024-04-02T16:12:14.729153+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705456+0000","last_change":"2024-04-02T16:12:14.705456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705456+0000","last_undegraded":"2024-04-02T16:12:14.705456+0000","last_fullsized":"2024-04-02T16:12:14.705456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737546+0000","last_change":"2024-04-02T16:12:14.735767+0000","last_active":"2024-04-02T16:12:14.737546+0000","last_peered":"2024-04-02T16:12:14.737546+0000","last_clean":"2024-04-02T16:12:14.737546+0000","last_became_active":"2024-04-02T16:12:14.735499+0000","last_became_peered":"2024-04-02T16:12:14.735499+0000","last_unstale":"2024-04-02T16:12:14.737546+0000","last_undegraded":"2024-04-02T16:12:14.737546+0000","last_fullsized":"2024-04-02T16:12:14.737546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709624+0000","last_change":"2024-04-02T16:12:14.709624+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709624+0000","last_undegraded":"2024-04-02T16:12:14.709624+0000","last_fullsized":"2024-04-02T16:12:14.709624+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.716043+0000","last_change":"2024-04-02T16:12:14.714641+0000","last_active":"2024-04-02T16:12:14.716043+0000","last_peered":"2024-04-02T16:12:14.716043+0000","last_clean":"2024-04-02T16:12:14.716043+0000","last_became_active":"2024-04-02T16:12:14.714242+0000","last_became_peered":"2024-04-02T16:12:14.714242+0000","last_unstale":"2024-04-02T16:12:14.716043+0000","last_undegraded":"2024-04-02T16:12:14.716043+0000","last_fullsized":"2024-04-02T16:12:14.716043+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703317+0000","last_change":"2024-04-02T16:12:14.703317+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703317+0000","last_undegraded":"2024-04-02T16:12:14.703317+0000","last_fullsized":"2024-04-02T16:12:14.703317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.717058+0000","last_change":"2024-04-02T16:12:14.711749+0000","last_active":"2024-04-02T16:12:14.717058+0000","last_peered":"2024-04-02T16:12:14.717058+0000","last_clean":"2024-04-02T16:12:14.717058+0000","last_became_active":"2024-04-02T16:12:14.711308+0000","last_became_peered":"2024-04-02T16:12:14.711308+0000","last_unstale":"2024-04-02T16:12:14.717058+0000","last_undegraded":"2024-04-02T16:12:14.717058+0000","last_fullsized":"2024-04-02T16:12:14.717058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.708089+0000","last_change":"2024-04-02T16:12:14.708089+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.708089+0000","last_undegraded":"2024-04-02T16:12:14.708089+0000","last_fullsized":"2024-04-02T16:12:14.708089+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734463+0000","last_change":"2024-04-02T16:12:14.728114+0000","last_active":"2024-04-02T16:12:14.734463+0000","last_peered":"2024-04-02T16:12:14.734463+0000","last_clean":"2024-04-02T16:12:14.734463+0000","last_became_active":"2024-04-02T16:12:14.727429+0000","last_became_peered":"2024-04-02T16:12:14.727429+0000","last_unstale":"2024-04-02T16:12:14.734463+0000","last_undegraded":"2024-04-02T16:12:14.734463+0000","last_fullsized":"2024-04-02T16:12:14.734463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:05.668055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.709940+0000","last_change":"2024-04-02T16:12:14.709940+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.709940+0000","last_undegraded":"2024-04-02T16:12:14.709940+0000","last_fullsized":"2024-04-02T16:12:14.709940+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722851+0000","last_change":"2024-04-02T16:12:14.716172+0000","last_active":"2024-04-02T16:12:14.722851+0000","last_peered":"2024-04-02T16:12:14.722851+0000","last_clean":"2024-04-02T16:12:14.722851+0000","last_became_active":"2024-04-02T16:12:14.714967+0000","last_became_peered":"2024-04-02T16:12:14.714967+0000","last_unstale":"2024-04-02T16:12:14.722851+0000","last_undegraded":"2024-04-02T16:12:14.722851+0000","last_fullsized":"2024-04-02T16:12:14.722851+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:19.185664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697302+0000","last_change":"2024-04-02T16:12:14.697302+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697302+0000","last_undegraded":"2024-04-02T16:12:14.697302+0000","last_fullsized":"2024-04-02T16:12:14.697302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.722096+0000","last_change":"2024-04-02T16:12:14.718370+0000","last_active":"2024-04-02T16:12:14.722096+0000","last_peered":"2024-04-02T16:12:14.722096+0000","last_clean":"2024-04-02T16:12:14.722096+0000","last_became_active":"2024-04-02T16:12:14.717749+0000","last_became_peered":"2024-04-02T16:12:14.717749+0000","last_unstale":"2024-04-02T16:12:14.722096+0000","last_undegraded":"2024-04-02T16:12:14.722096+0000","last_fullsized":"2024-04-02T16:12:14.722096+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:19:34.428530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697456+0000","last_change":"2024-04-02T16:12:14.697456+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697456+0000","last_undegraded":"2024-04-02T16:12:14.697456+0000","last_fullsized":"2024-04-02T16:12:14.697456+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718780+0000","last_change":"2024-04-02T16:12:14.717818+0000","last_active":"2024-04-02T16:12:14.718780+0000","last_peered":"2024-04-02T16:12:14.718780+0000","last_clean":"2024-04-02T16:12:14.718780+0000","last_became_active":"2024-04-02T16:12:14.717461+0000","last_became_peered":"2024-04-02T16:12:14.717461+0000","last_unstale":"2024-04-02T16:12:14.718780+0000","last_undegraded":"2024-04-02T16:12:14.718780+0000","last_fullsized":"2024-04-02T16:12:14.718780+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697306+0000","last_change":"2024-04-02T16:12:14.697306+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697306+0000","last_undegraded":"2024-04-02T16:12:14.697306+0000","last_fullsized":"2024-04-02T16:12:14.697306+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734053+0000","last_change":"2024-04-02T16:12:14.729387+0000","last_active":"2024-04-02T16:12:14.734053+0000","last_peered":"2024-04-02T16:12:14.734053+0000","last_clean":"2024-04-02T16:12:14.734053+0000","last_became_active":"2024-04-02T16:12:14.728744+0000","last_became_peered":"2024-04-02T16:12:14.728744+0000","last_unstale":"2024-04-02T16:12:14.734053+0000","last_undegraded":"2024-04-02T16:12:14.734053+0000","last_fullsized":"2024-04-02T16:12:14.734053+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697542+0000","last_change":"2024-04-02T16:12:14.697542+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697542+0000","last_undegraded":"2024-04-02T16:12:14.697542+0000","last_fullsized":"2024-04-02T16:12:14.697542+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.733963+0000","last_change":"2024-04-02T16:12:14.728831+0000","last_active":"2024-04-02T16:12:14.733963+0000","last_peered":"2024-04-02T16:12:14.733963+0000","last_clean":"2024-04-02T16:12:14.733963+0000","last_became_active":"2024-04-02T16:12:14.727741+0000","last_became_peered":"2024-04-02T16:12:14.727741+0000","last_unstale":"2024-04-02T16:12:14.733963+0000","last_undegraded":"2024-04-02T16:12:14.733963+0000","last_fullsized":"2024-04-02T16:12:14.733963+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:35:51.046626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.710015+0000","last_change":"2024-04-02T16:12:14.710015+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.710015+0000","last_undegraded":"2024-04-02T16:12:14.710015+0000","last_fullsized":"2024-04-02T16:12:14.710015+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.740130+0000","last_change":"2024-04-02T16:12:14.737907+0000","last_active":"2024-04-02T16:12:14.740130+0000","last_peered":"2024-04-02T16:12:14.740130+0000","last_clean":"2024-04-02T16:12:14.740130+0000","last_became_active":"2024-04-02T16:12:14.737140+0000","last_became_peered":"2024-04-02T16:12:14.737140+0000","last_unstale":"2024-04-02T16:12:14.740130+0000","last_undegraded":"2024-04-02T16:12:14.740130+0000","last_fullsized":"2024-04-02T16:12:14.740130+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.697201+0000","last_change":"2024-04-02T16:12:14.697201+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.697201+0000","last_undegraded":"2024-04-02T16:12:14.697201+0000","last_fullsized":"2024-04-02T16:12:14.697201+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729195+0000","last_change":"2024-04-02T16:12:14.725690+0000","last_active":"2024-04-02T16:12:14.729195+0000","last_peered":"2024-04-02T16:12:14.729195+0000","last_clean":"2024-04-02T16:12:14.729195+0000","last_became_active":"2024-04-02T16:12:14.725425+0000","last_became_peered":"2024-04-02T16:12:14.725425+0000","last_unstale":"2024-04-02T16:12:14.729195+0000","last_undegraded":"2024-04-02T16:12:14.729195+0000","last_fullsized":"2024-04-02T16:12:14.729195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.706947+0000","last_change":"2024-04-02T16:12:14.706947+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.706947+0000","last_undegraded":"2024-04-02T16:12:14.706947+0000","last_fullsized":"2024-04-02T16:12:14.706947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.713765+0000","last_change":"2024-04-02T16:12:14.711273+0000","last_active":"2024-04-02T16:12:14.713765+0000","last_peered":"2024-04-02T16:12:14.713765+0000","last_clean":"2024-04-02T16:12:14.713765+0000","last_became_active":"2024-04-02T16:12:14.710880+0000","last_became_peered":"2024-04-02T16:12:14.710880+0000","last_unstale":"2024-04-02T16:12:14.713765+0000","last_undegraded":"2024-04-02T16:12:14.713765+0000","last_fullsized":"2024-04-02T16:12:14.713765+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.700493+0000","last_change":"2024-04-02T16:12:14.700493+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.700493+0000","last_undegraded":"2024-04-02T16:12:14.700493+0000","last_fullsized":"2024-04-02T16:12:14.700493+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.718705+0000","last_change":"2024-04-02T16:12:14.716204+0000","last_active":"2024-04-02T16:12:14.718705+0000","last_peered":"2024-04-02T16:12:14.718705+0000","last_clean":"2024-04-02T16:12:14.718705+0000","last_became_active":"2024-04-02T16:12:14.715693+0000","last_became_peered":"2024-04-02T16:12:14.715693+0000","last_unstale":"2024-04-02T16:12:14.718705+0000","last_undegraded":"2024-04-02T16:12:14.718705+0000","last_fullsized":"2024-04-02T16:12:14.718705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.711442+0000","last_change":"2024-04-02T16:12:14.711442+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.711442+0000","last_undegraded":"2024-04-02T16:12:14.711442+0000","last_fullsized":"2024-04-02T16:12:14.711442+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.726471+0000","last_change":"2024-04-02T16:12:14.723590+0000","last_active":"2024-04-02T16:12:14.726471+0000","last_peered":"2024-04-02T16:12:14.726471+0000","last_clean":"2024-04-02T16:12:14.726471+0000","last_became_active":"2024-04-02T16:12:14.723145+0000","last_became_peered":"2024-04-02T16:12:14.723145+0000","last_unstale":"2024-04-02T16:12:14.726471+0000","last_undegraded":"2024-04-02T16:12:14.726471+0000","last_fullsized":"2024-04-02T16:12:14.726471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.703745+0000","last_change":"2024-04-02T16:12:14.703745+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.703745+0000","last_undegraded":"2024-04-02T16:12:14.703745+0000","last_fullsized":"2024-04-02T16:12:14.703745+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.729200+0000","last_change":"2024-04-02T16:12:14.725705+0000","last_active":"2024-04-02T16:12:14.729200+0000","last_peered":"2024-04-02T16:12:14.729200+0000","last_clean":"2024-04-02T16:12:14.729200+0000","last_became_active":"2024-04-02T16:12:14.725450+0000","last_became_peered":"2024-04-02T16:12:14.725450+0000","last_unstale":"2024-04-02T16:12:14.729200+0000","last_undegraded":"2024-04-02T16:12:14.729200+0000","last_fullsized":"2024-04-02T16:12:14.729200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.702914+0000","last_change":"2024-04-02T16:12:14.702914+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.702914+0000","last_undegraded":"2024-04-02T16:12:14.702914+0000","last_fullsized":"2024-04-02T16:12:14.702914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.734476+0000","last_change":"2024-04-02T16:12:14.726690+0000","last_active":"2024-04-02T16:12:14.734476+0000","last_peered":"2024-04-02T16:12:14.734476+0000","last_clean":"2024-04-02T16:12:14.734476+0000","last_became_active":"2024-04-02T16:12:14.725905+0000","last_became_peered":"2024-04-02T16:12:14.725905+0000","last_unstale":"2024-04-02T16:12:14.734476+0000","last_undegraded":"2024-04-02T16:12:14.734476+0000","last_fullsized":"2024-04-02T16:12:14.734476+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:54.536550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.704293+0000","last_change":"2024-04-02T16:12:14.704293+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.704293+0000","last_undegraded":"2024-04-02T16:12:14.704293+0000","last_fullsized":"2024-04-02T16:12:14.704293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737552+0000","last_change":"2024-04-02T16:12:14.736388+0000","last_active":"2024-04-02T16:12:14.737552+0000","last_peered":"2024-04-02T16:12:14.737552+0000","last_clean":"2024-04-02T16:12:14.737552+0000","last_became_active":"2024-04-02T16:12:14.736096+0000","last_became_peered":"2024-04-02T16:12:14.736096+0000","last_unstale":"2024-04-02T16:12:14.737552+0000","last_undegraded":"2024-04-02T16:12:14.737552+0000","last_fullsized":"2024-04-02T16:12:14.737552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":27,"state":"creating+peering","last_fresh":"2024-04-02T16:12:14.705554+0000","last_change":"2024-04-02T16:12:14.705554+0000","last_active":"2024-04-02T16:12:14.679647+0000","last_peered":"2024-04-02T16:12:14.679647+0000","last_clean":"2024-04-02T16:12:14.679647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-02T16:12:14.705554+0000","last_undegraded":"2024-04-02T16:12:14.705554+0000","last_fullsized":"2024-04-02T16:12:14.705554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.737786+0000","last_change":"2024-04-02T16:12:14.735597+0000","last_active":"2024-04-02T16:12:14.737786+0000","last_peered":"2024-04-02T16:12:14.737786+0000","last_clean":"2024-04-02T16:12:14.737786+0000","last_became_active":"2024-04-02T16:12:14.734832+0000","last_became_peered":"2024-04-02T16:12:14.734832+0000","last_unstale":"2024-04-02T16:12:14.737786+0000","last_undegraded":"2024-04-02T16:12:14.737786+0000","last_fullsized":"2024-04-02T16:12:14.737786+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-02T16:12:14.720269+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:14.720269+0000","last_peered":"2024-04-02T16:12:14.720269+0000","last_clean":"2024-04-02T16:12:14.720269+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:14.720269+0000","last_undegraded":"2024-04-02T16:12:14.720269+0000","last_fullsized":"2024-04-02T16:12:14.720269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":695038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:23.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:23.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 Processor -- start 2024-04-02T16:12:23.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 -- start start 2024-04-02T16:12:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f97400ae080 0x7f97400b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97400b3210 con 0x7f97400b2780 2024-04-02T16:12:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974006db10 con 0x7f97400b1c10 2024-04-02T16:12:23.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f974490e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f97400ae460 con 0x7f97400ae080 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f97400b2780 0x7f97401522a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55942/0 (socket says 172.21.15.40:55942) 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 -- 172.21.15.40:0/3032908684 learned_addr learned my addr 172.21.15.40:0/3032908684 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973e59c640 1 --2- 172.21.15.40:0/3032908684 >> v2:172.21.15.204:3300/0 conn(0x7f97400ae080 0x7f97400b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036705940 0 0) 0x7f97400b3210 con 0x7f97400b2780 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f972c003530 con 0x7f97400b2780 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1182386790 0 0) 0x7f972c003530 con 0x7f97400b2780 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 >> v2:172.21.15.204:3300/0 conn(0x7f97400ae080 msgr2=0x7f97400b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 --2- 172.21.15.40:0/3032908684 >> v2:172.21.15.204:3300/0 conn(0x7f97400ae080 0x7f97400b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 >> v1:172.21.15.204:6789/0 conn(0x7f97400b1c10 legacy=0x7f97400b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97400af670 con 0x7f97400b2780 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f9738001a20 con 0x7f97400b2780 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 >> v1:172.21.15.40:6789/0 conn(0x7f97400b2780 legacy=0x7f97401522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f972c003a10 0x7f972c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97400ae460 con 0x7f972c0051d0 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f972c00c0d0 con 0x7f972c003a10 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f972c00c2d0 con 0x7f972c004810 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973e59c640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f972c003a10 0x7f972c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973dd9b640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 -- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f972c004810 msgr2=0x7f972c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 -- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f972c003a10 msgr2=0x7f972c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f972c003a10 0x7f972c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973ed9d640 1 -- 172.21.15.40:0/3032908684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c003530 con 0x7f972c0051d0 2024-04-02T16:12:23.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.392+0000 7f973dd9b640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f972c004810 0x7f972c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f973ed9d640 1 -- 172.21.15.40:0/3032908684 reap_dead start 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f973ed9d640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97400b33e0 tx=0x7f973801d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ba40660cf7e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800a3b0 con 0x7f972c0051d0 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97380ac620 con 0x7f972c0051d0 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f973d59a640 1 -- 172.21.15.40:0/3032908684 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97380aca10 con 0x7f972c0051d0 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 msgr2=0x7f972c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97400b33e0 tx=0x7f973801d240 comp rx=0 tx=0).stop 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- 172.21.15.40:0/3032908684 shutdown_connections 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 --2- 172.21.15.40:0/3032908684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f972c00b9d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- 172.21.15.40:0/3032908684 >> 172.21.15.40:0/3032908684 conn(0x7f97400b8fd0 msgr2=0x7f97400b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- 172.21.15.40:0/3032908684 shutdown_connections 2024-04-02T16:12:23.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- 172.21.15.40:0/3032908684 wait complete. 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 Processor -- start 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- start start 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974009be40 0x7f97401c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97401c6980 0x7f97401c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97400ae780 con 0x7f972c0051d0 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97400afde0 con 0x7f974009be40 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.396+0000 7f974490e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f97400af2f0 con 0x7f97401c6980 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59342/0 (socket says 172.21.15.40:59342) 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 -- 172.21.15.40:0/204264891 learned_addr learned my addr 172.21.15.40:0/204264891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973dd9b640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974009be40 0x7f97401c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973ed9d640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97401c6980 0x7f97401c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 -- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97401c6980 msgr2=0x7f97401c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97401c6980 0x7f97401c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 -- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974009be40 msgr2=0x7f97401c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974009be40 0x7f97401c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 -- 172.21.15.40:0/204264891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401c73c0 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973e59c640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9728011590 tx=0x7f9728011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728015870 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728015a10 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97401c9920 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728015dc0 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974006fe40 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97400685f0 con 0x7f972c0051d0 2024-04-02T16:12:23.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9728013400 con 0x7f972c0051d0 2024-04-02T16:12:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 0x7f972006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f9728087200 con 0x7f972c0051d0 2024-04-02T16:12:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.400+0000 7f973dd9b640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 0x7f972006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.404+0000 7f973dd9b640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 0x7f972006a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f974009d920 tx=0x7f9730009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.404+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f9728019030 con 0x7f972c0051d0 2024-04-02T16:12:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.564+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97400b71a0 con 0x7f97200683b0 2024-04-02T16:12:23.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.584+0000 7f9736ffd640 1 -- 172.21.15.40:0/204264891 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251963 (secure 0 0 0) 0x7f97400b71a0 con 0x7f97200683b0 2024-04-02T16:12:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 msgr2=0x7f972006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 0x7f972006a870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f974009d920 tx=0x7f9730009040 comp rx=0 tx=0).stop 2024-04-02T16:12:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 msgr2=0x7f97401e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9728011590 tx=0x7f9728011a50 comp rx=0 tx=0).stop 2024-04-02T16:12:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 shutdown_connections 2024-04-02T16:12:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97401c6980 0x7f97401c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f97200683b0 0x7f972006a870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f974009be40 0x7f97401c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 --2- 172.21.15.40:0/204264891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c0051d0 0x7f97401e0100 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 >> 172.21.15.40:0/204264891 conn(0x7f97400b8fd0 msgr2=0x7f97401cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 shutdown_connections 2024-04-02T16:12:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.588+0000 7f974490e640 1 -- 172.21.15.40:0/204264891 wait complete. 2024-04-02T16:12:23.607 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-04-02T16:12:21.864744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224104,"kb_used_data":3752,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750616,"statfs":{"total":773094113280,"available":772864630784,"internally_reserved":0,"allocated":3842048,"data_stored":2177652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.170028"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575172+0000","last_change":"2024-04-02T16:12:15.753715+0000","last_active":"2024-04-02T16:12:17.575172+0000","last_peered":"2024-04-02T16:12:17.575172+0000","last_clean":"2024-04-02T16:12:17.575172+0000","last_became_active":"2024-04-02T16:12:15.752415+0000","last_became_peered":"2024-04-02T16:12:15.752415+0000","last_unstale":"2024-04-02T16:12:17.575172+0000","last_undegraded":"2024-04-02T16:12:17.575172+0000","last_fullsized":"2024-04-02T16:12:17.575172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:23:43.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577091+0000","last_change":"2024-04-02T16:12:15.713490+0000","last_active":"2024-04-02T16:12:17.577091+0000","last_peered":"2024-04-02T16:12:17.577091+0000","last_clean":"2024-04-02T16:12:17.577091+0000","last_became_active":"2024-04-02T16:12:15.712802+0000","last_became_peered":"2024-04-02T16:12:15.712802+0000","last_unstale":"2024-04-02T16:12:17.577091+0000","last_undegraded":"2024-04-02T16:12:17.577091+0000","last_fullsized":"2024-04-02T16:12:17.577091+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:37:13.746991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703204+0000","last_change":"2024-04-02T16:12:15.721991+0000","last_active":"2024-04-02T16:12:16.703204+0000","last_peered":"2024-04-02T16:12:16.703204+0000","last_clean":"2024-04-02T16:12:16.703204+0000","last_became_active":"2024-04-02T16:12:15.717250+0000","last_became_peered":"2024-04-02T16:12:15.717250+0000","last_unstale":"2024-04-02T16:12:16.703204+0000","last_undegraded":"2024-04-02T16:12:16.703204+0000","last_fullsized":"2024-04-02T16:12:16.703204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:37:06.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573809+0000","last_change":"2024-04-02T16:12:15.731872+0000","last_active":"2024-04-02T16:12:17.573809+0000","last_peered":"2024-04-02T16:12:17.573809+0000","last_clean":"2024-04-02T16:12:17.573809+0000","last_became_active":"2024-04-02T16:12:15.731555+0000","last_became_peered":"2024-04-02T16:12:15.731555+0000","last_unstale":"2024-04-02T16:12:17.573809+0000","last_undegraded":"2024-04-02T16:12:17.573809+0000","last_fullsized":"2024-04-02T16:12:17.573809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:38:30.832068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.698832+0000","last_change":"2024-04-02T16:12:15.715003+0000","last_active":"2024-04-02T16:12:16.698832+0000","last_peered":"2024-04-02T16:12:16.698832+0000","last_clean":"2024-04-02T16:12:16.698832+0000","last_became_active":"2024-04-02T16:12:15.714505+0000","last_became_peered":"2024-04-02T16:12:15.714505+0000","last_unstale":"2024-04-02T16:12:16.698832+0000","last_undegraded":"2024-04-02T16:12:16.698832+0000","last_fullsized":"2024-04-02T16:12:16.698832+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:08.589740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576290+0000","last_change":"2024-04-02T16:12:15.740291+0000","last_active":"2024-04-02T16:12:17.576290+0000","last_peered":"2024-04-02T16:12:17.576290+0000","last_clean":"2024-04-02T16:12:17.576290+0000","last_became_active":"2024-04-02T16:12:15.739989+0000","last_became_peered":"2024-04-02T16:12:15.739989+0000","last_unstale":"2024-04-02T16:12:17.576290+0000","last_undegraded":"2024-04-02T16:12:17.576290+0000","last_fullsized":"2024-04-02T16:12:17.576290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:37:06.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573883+0000","last_change":"2024-04-02T16:12:15.746868+0000","last_active":"2024-04-02T16:12:17.573883+0000","last_peered":"2024-04-02T16:12:17.573883+0000","last_clean":"2024-04-02T16:12:17.573883+0000","last_became_active":"2024-04-02T16:12:15.746149+0000","last_became_peered":"2024-04-02T16:12:15.746149+0000","last_unstale":"2024-04-02T16:12:17.573883+0000","last_undegraded":"2024-04-02T16:12:17.573883+0000","last_fullsized":"2024-04-02T16:12:17.573883+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:03:12.472070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573383+0000","last_change":"2024-04-02T16:12:15.746386+0000","last_active":"2024-04-02T16:12:17.573383+0000","last_peered":"2024-04-02T16:12:17.573383+0000","last_clean":"2024-04-02T16:12:17.573383+0000","last_became_active":"2024-04-02T16:12:15.744934+0000","last_became_peered":"2024-04-02T16:12:15.744934+0000","last_unstale":"2024-04-02T16:12:17.573383+0000","last_undegraded":"2024-04-02T16:12:17.573383+0000","last_fullsized":"2024-04-02T16:12:17.573383+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:09:18.120539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704992+0000","last_change":"2024-04-02T16:12:15.712224+0000","last_active":"2024-04-02T16:12:16.704992+0000","last_peered":"2024-04-02T16:12:16.704992+0000","last_clean":"2024-04-02T16:12:16.704992+0000","last_became_active":"2024-04-02T16:12:15.711484+0000","last_became_peered":"2024-04-02T16:12:15.711484+0000","last_unstale":"2024-04-02T16:12:16.704992+0000","last_undegraded":"2024-04-02T16:12:16.704992+0000","last_fullsized":"2024-04-02T16:12:16.704992+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:58:02.033067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574862+0000","last_change":"2024-04-02T16:12:15.718104+0000","last_active":"2024-04-02T16:12:17.574862+0000","last_peered":"2024-04-02T16:12:17.574862+0000","last_clean":"2024-04-02T16:12:17.574862+0000","last_became_active":"2024-04-02T16:12:15.716718+0000","last_became_peered":"2024-04-02T16:12:15.716718+0000","last_unstale":"2024-04-02T16:12:17.574862+0000","last_undegraded":"2024-04-02T16:12:17.574862+0000","last_fullsized":"2024-04-02T16:12:17.574862+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:03:12.472070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574085+0000","last_change":"2024-04-02T16:12:15.735883+0000","last_active":"2024-04-02T16:12:17.574085+0000","last_peered":"2024-04-02T16:12:17.574085+0000","last_clean":"2024-04-02T16:12:17.574085+0000","last_became_active":"2024-04-02T16:12:15.731391+0000","last_became_peered":"2024-04-02T16:12:15.731391+0000","last_unstale":"2024-04-02T16:12:17.574085+0000","last_undegraded":"2024-04-02T16:12:17.574085+0000","last_fullsized":"2024-04-02T16:12:17.574085+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:37:06.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702552+0000","last_change":"2024-04-02T16:12:15.722131+0000","last_active":"2024-04-02T16:12:16.702552+0000","last_peered":"2024-04-02T16:12:16.702552+0000","last_clean":"2024-04-02T16:12:16.702552+0000","last_became_active":"2024-04-02T16:12:15.721274+0000","last_became_peered":"2024-04-02T16:12:15.721274+0000","last_unstale":"2024-04-02T16:12:16.702552+0000","last_undegraded":"2024-04-02T16:12:16.702552+0000","last_fullsized":"2024-04-02T16:12:16.702552+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:03:12.472070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.699248+0000","last_change":"2024-04-02T16:12:15.715584+0000","last_active":"2024-04-02T16:12:16.699248+0000","last_peered":"2024-04-02T16:12:16.699248+0000","last_clean":"2024-04-02T16:12:16.699248+0000","last_became_active":"2024-04-02T16:12:15.715293+0000","last_became_peered":"2024-04-02T16:12:15.715293+0000","last_unstale":"2024-04-02T16:12:16.699248+0000","last_undegraded":"2024-04-02T16:12:16.699248+0000","last_fullsized":"2024-04-02T16:12:16.699248+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:39.899139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576704+0000","last_change":"2024-04-02T16:12:15.711878+0000","last_active":"2024-04-02T16:12:17.576704+0000","last_peered":"2024-04-02T16:12:17.576704+0000","last_clean":"2024-04-02T16:12:17.576704+0000","last_became_active":"2024-04-02T16:12:15.711570+0000","last_became_peered":"2024-04-02T16:12:15.711570+0000","last_unstale":"2024-04-02T16:12:17.576704+0000","last_undegraded":"2024-04-02T16:12:17.576704+0000","last_fullsized":"2024-04-02T16:12:17.576704+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:49:34.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701559+0000","last_change":"2024-04-02T16:12:15.739999+0000","last_active":"2024-04-02T16:12:16.701559+0000","last_peered":"2024-04-02T16:12:16.701559+0000","last_clean":"2024-04-02T16:12:16.701559+0000","last_became_active":"2024-04-02T16:12:15.739261+0000","last_became_peered":"2024-04-02T16:12:15.739261+0000","last_unstale":"2024-04-02T16:12:16.701559+0000","last_undegraded":"2024-04-02T16:12:16.701559+0000","last_fullsized":"2024-04-02T16:12:16.701559+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:02:08.209108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573985+0000","last_change":"2024-04-02T16:12:15.746971+0000","last_active":"2024-04-02T16:12:17.573985+0000","last_peered":"2024-04-02T16:12:17.573985+0000","last_clean":"2024-04-02T16:12:17.573985+0000","last_became_active":"2024-04-02T16:12:15.745760+0000","last_became_peered":"2024-04-02T16:12:15.745760+0000","last_unstale":"2024-04-02T16:12:17.573985+0000","last_undegraded":"2024-04-02T16:12:17.573985+0000","last_fullsized":"2024-04-02T16:12:17.573985+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:42.692500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574838+0000","last_change":"2024-04-02T16:12:15.739670+0000","last_active":"2024-04-02T16:12:17.574838+0000","last_peered":"2024-04-02T16:12:17.574838+0000","last_clean":"2024-04-02T16:12:17.574838+0000","last_became_active":"2024-04-02T16:12:15.738937+0000","last_became_peered":"2024-04-02T16:12:15.738937+0000","last_unstale":"2024-04-02T16:12:17.574838+0000","last_undegraded":"2024-04-02T16:12:17.574838+0000","last_fullsized":"2024-04-02T16:12:17.574838+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:49:34.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577326+0000","last_change":"2024-04-02T16:12:15.713140+0000","last_active":"2024-04-02T16:12:17.577326+0000","last_peered":"2024-04-02T16:12:17.577326+0000","last_clean":"2024-04-02T16:12:17.577326+0000","last_became_active":"2024-04-02T16:12:15.712343+0000","last_became_peered":"2024-04-02T16:12:15.712343+0000","last_unstale":"2024-04-02T16:12:17.577326+0000","last_undegraded":"2024-04-02T16:12:17.577326+0000","last_fullsized":"2024-04-02T16:12:17.577326+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:42:40.317670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577599+0000","last_change":"2024-04-02T16:12:15.715504+0000","last_active":"2024-04-02T16:12:17.577599+0000","last_peered":"2024-04-02T16:12:17.577599+0000","last_clean":"2024-04-02T16:12:17.577599+0000","last_became_active":"2024-04-02T16:12:15.715190+0000","last_became_peered":"2024-04-02T16:12:15.715190+0000","last_unstale":"2024-04-02T16:12:17.577599+0000","last_undegraded":"2024-04-02T16:12:17.577599+0000","last_fullsized":"2024-04-02T16:12:17.577599+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:12:39.949913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.698810+0000","last_change":"2024-04-02T16:12:15.741835+0000","last_active":"2024-04-02T16:12:16.698810+0000","last_peered":"2024-04-02T16:12:16.698810+0000","last_clean":"2024-04-02T16:12:16.698810+0000","last_became_active":"2024-04-02T16:12:15.741576+0000","last_became_peered":"2024-04-02T16:12:15.741576+0000","last_unstale":"2024-04-02T16:12:16.698810+0000","last_undegraded":"2024-04-02T16:12:16.698810+0000","last_fullsized":"2024-04-02T16:12:16.698810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:09:34.443357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701556+0000","last_change":"2024-04-02T16:12:15.713788+0000","last_active":"2024-04-02T16:12:16.701556+0000","last_peered":"2024-04-02T16:12:16.701556+0000","last_clean":"2024-04-02T16:12:16.701556+0000","last_became_active":"2024-04-02T16:12:15.712301+0000","last_became_peered":"2024-04-02T16:12:15.712301+0000","last_unstale":"2024-04-02T16:12:16.701556+0000","last_undegraded":"2024-04-02T16:12:16.701556+0000","last_fullsized":"2024-04-02T16:12:16.701556+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:34:26.572656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701917+0000","last_change":"2024-04-02T16:12:15.738995+0000","last_active":"2024-04-02T16:12:16.701917+0000","last_peered":"2024-04-02T16:12:16.701917+0000","last_clean":"2024-04-02T16:12:16.701917+0000","last_became_active":"2024-04-02T16:12:15.733310+0000","last_became_peered":"2024-04-02T16:12:15.733310+0000","last_unstale":"2024-04-02T16:12:16.701917+0000","last_undegraded":"2024-04-02T16:12:16.701917+0000","last_fullsized":"2024-04-02T16:12:16.701917+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:23:43.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.572662+0000","last_change":"2024-04-02T16:12:15.725753+0000","last_active":"2024-04-02T16:12:17.572662+0000","last_peered":"2024-04-02T16:12:17.572662+0000","last_clean":"2024-04-02T16:12:17.572662+0000","last_became_active":"2024-04-02T16:12:15.725449+0000","last_became_peered":"2024-04-02T16:12:15.725449+0000","last_unstale":"2024-04-02T16:12:17.572662+0000","last_undegraded":"2024-04-02T16:12:17.572662+0000","last_fullsized":"2024-04-02T16:12:17.572662+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:23:43.218533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701180+0000","last_change":"2024-04-02T16:12:15.725768+0000","last_active":"2024-04-02T16:12:16.701180+0000","last_peered":"2024-04-02T16:12:16.701180+0000","last_clean":"2024-04-02T16:12:16.701180+0000","last_became_active":"2024-04-02T16:12:15.717154+0000","last_became_peered":"2024-04-02T16:12:15.717154+0000","last_unstale":"2024-04-02T16:12:16.701180+0000","last_undegraded":"2024-04-02T16:12:16.701180+0000","last_fullsized":"2024-04-02T16:12:16.701180+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:20:31.605127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575150+0000","last_change":"2024-04-02T16:12:15.719001+0000","last_active":"2024-04-02T16:12:17.575150+0000","last_peered":"2024-04-02T16:12:17.575150+0000","last_clean":"2024-04-02T16:12:17.575150+0000","last_became_active":"2024-04-02T16:12:15.718649+0000","last_became_peered":"2024-04-02T16:12:15.718649+0000","last_unstale":"2024-04-02T16:12:17.575150+0000","last_undegraded":"2024-04-02T16:12:17.575150+0000","last_fullsized":"2024-04-02T16:12:17.575150+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:42.692500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577645+0000","last_change":"2024-04-02T16:12:15.719987+0000","last_active":"2024-04-02T16:12:17.577645+0000","last_peered":"2024-04-02T16:12:17.577645+0000","last_clean":"2024-04-02T16:12:17.577645+0000","last_became_active":"2024-04-02T16:12:15.719526+0000","last_became_peered":"2024-04-02T16:12:15.719526+0000","last_unstale":"2024-04-02T16:12:17.577645+0000","last_undegraded":"2024-04-02T16:12:17.577645+0000","last_fullsized":"2024-04-02T16:12:17.577645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:45:51.579867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574410+0000","last_change":"2024-04-02T16:12:15.731815+0000","last_active":"2024-04-02T16:12:17.574410+0000","last_peered":"2024-04-02T16:12:17.574410+0000","last_clean":"2024-04-02T16:12:17.574410+0000","last_became_active":"2024-04-02T16:12:15.731516+0000","last_became_peered":"2024-04-02T16:12:15.731516+0000","last_unstale":"2024-04-02T16:12:17.574410+0000","last_undegraded":"2024-04-02T16:12:17.574410+0000","last_fullsized":"2024-04-02T16:12:17.574410+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:47:10.119028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702966+0000","last_change":"2024-04-02T16:12:15.716963+0000","last_active":"2024-04-02T16:12:16.702966+0000","last_peered":"2024-04-02T16:12:16.702966+0000","last_clean":"2024-04-02T16:12:16.702966+0000","last_became_active":"2024-04-02T16:12:15.715956+0000","last_became_peered":"2024-04-02T16:12:15.715956+0000","last_unstale":"2024-04-02T16:12:16.702966+0000","last_undegraded":"2024-04-02T16:12:16.702966+0000","last_fullsized":"2024-04-02T16:12:16.702966+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:27.972798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703554+0000","last_change":"2024-04-02T16:12:15.749802+0000","last_active":"2024-04-02T16:12:16.703554+0000","last_peered":"2024-04-02T16:12:16.703554+0000","last_clean":"2024-04-02T16:12:16.703554+0000","last_became_active":"2024-04-02T16:12:15.732319+0000","last_became_peered":"2024-04-02T16:12:15.732319+0000","last_unstale":"2024-04-02T16:12:16.703554+0000","last_undegraded":"2024-04-02T16:12:16.703554+0000","last_fullsized":"2024-04-02T16:12:16.703554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:48:51.548990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573557+0000","last_change":"2024-04-02T16:12:15.740578+0000","last_active":"2024-04-02T16:12:17.573557+0000","last_peered":"2024-04-02T16:12:17.573557+0000","last_clean":"2024-04-02T16:12:17.573557+0000","last_became_active":"2024-04-02T16:12:15.739330+0000","last_became_peered":"2024-04-02T16:12:15.739330+0000","last_unstale":"2024-04-02T16:12:17.573557+0000","last_undegraded":"2024-04-02T16:12:17.573557+0000","last_fullsized":"2024-04-02T16:12:17.573557+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:20:31.605127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574596+0000","last_change":"2024-04-02T16:12:15.723477+0000","last_active":"2024-04-02T16:12:17.574596+0000","last_peered":"2024-04-02T16:12:17.574596+0000","last_clean":"2024-04-02T16:12:17.574596+0000","last_became_active":"2024-04-02T16:12:15.722558+0000","last_became_peered":"2024-04-02T16:12:15.722558+0000","last_unstale":"2024-04-02T16:12:17.574596+0000","last_undegraded":"2024-04-02T16:12:17.574596+0000","last_fullsized":"2024-04-02T16:12:17.574596+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:26:37.860090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573469+0000","last_change":"2024-04-02T16:12:15.755962+0000","last_active":"2024-04-02T16:12:17.573469+0000","last_peered":"2024-04-02T16:12:17.573469+0000","last_clean":"2024-04-02T16:12:17.573469+0000","last_became_active":"2024-04-02T16:12:15.755430+0000","last_became_peered":"2024-04-02T16:12:15.755430+0000","last_unstale":"2024-04-02T16:12:17.573469+0000","last_undegraded":"2024-04-02T16:12:17.573469+0000","last_fullsized":"2024-04-02T16:12:17.573469+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:38:30.832068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573847+0000","last_change":"2024-04-02T16:12:15.731156+0000","last_active":"2024-04-02T16:12:17.573847+0000","last_peered":"2024-04-02T16:12:17.573847+0000","last_clean":"2024-04-02T16:12:17.573847+0000","last_became_active":"2024-04-02T16:12:15.730846+0000","last_became_peered":"2024-04-02T16:12:15.730846+0000","last_unstale":"2024-04-02T16:12:17.573847+0000","last_undegraded":"2024-04-02T16:12:17.573847+0000","last_fullsized":"2024-04-02T16:12:17.573847+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:31:03.977476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575415+0000","last_change":"2024-04-02T16:12:14.733795+0000","last_active":"2024-04-02T16:12:17.575415+0000","last_peered":"2024-04-02T16:12:17.575415+0000","last_clean":"2024-04-02T16:12:17.575415+0000","last_became_active":"2024-04-02T16:12:14.732992+0000","last_became_peered":"2024-04-02T16:12:14.732992+0000","last_unstale":"2024-04-02T16:12:17.575415+0000","last_undegraded":"2024-04-02T16:12:17.575415+0000","last_fullsized":"2024-04-02T16:12:17.575415+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575211+0000","last_change":"2024-04-02T16:12:15.753920+0000","last_active":"2024-04-02T16:12:17.575211+0000","last_peered":"2024-04-02T16:12:17.575211+0000","last_clean":"2024-04-02T16:12:17.575211+0000","last_became_active":"2024-04-02T16:12:15.752859+0000","last_became_peered":"2024-04-02T16:12:15.752859+0000","last_unstale":"2024-04-02T16:12:17.575211+0000","last_undegraded":"2024-04-02T16:12:17.575211+0000","last_fullsized":"2024-04-02T16:12:17.575211+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:48:51.548990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573993+0000","last_change":"2024-04-02T16:12:14.735269+0000","last_active":"2024-04-02T16:12:17.573993+0000","last_peered":"2024-04-02T16:12:17.573993+0000","last_clean":"2024-04-02T16:12:17.573993+0000","last_became_active":"2024-04-02T16:12:14.734879+0000","last_became_peered":"2024-04-02T16:12:14.734879+0000","last_unstale":"2024-04-02T16:12:17.573993+0000","last_undegraded":"2024-04-02T16:12:17.573993+0000","last_fullsized":"2024-04-02T16:12:17.573993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703786+0000","last_change":"2024-04-02T16:12:15.731905+0000","last_active":"2024-04-02T16:12:16.703786+0000","last_peered":"2024-04-02T16:12:16.703786+0000","last_clean":"2024-04-02T16:12:16.703786+0000","last_became_active":"2024-04-02T16:12:15.731168+0000","last_became_peered":"2024-04-02T16:12:15.731168+0000","last_unstale":"2024-04-02T16:12:16.703786+0000","last_undegraded":"2024-04-02T16:12:16.703786+0000","last_fullsized":"2024-04-02T16:12:16.703786+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:32.088733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576867+0000","last_change":"2024-04-02T16:12:14.711385+0000","last_active":"2024-04-02T16:12:17.576867+0000","last_peered":"2024-04-02T16:12:17.576867+0000","last_clean":"2024-04-02T16:12:17.576867+0000","last_became_active":"2024-04-02T16:12:14.711031+0000","last_became_peered":"2024-04-02T16:12:14.711031+0000","last_unstale":"2024-04-02T16:12:17.576867+0000","last_undegraded":"2024-04-02T16:12:17.576867+0000","last_fullsized":"2024-04-02T16:12:17.576867+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:48:50.538930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574817+0000","last_change":"2024-04-02T16:12:15.731617+0000","last_active":"2024-04-02T16:12:17.574817+0000","last_peered":"2024-04-02T16:12:17.574817+0000","last_clean":"2024-04-02T16:12:17.574817+0000","last_became_active":"2024-04-02T16:12:15.731205+0000","last_became_peered":"2024-04-02T16:12:15.731205+0000","last_unstale":"2024-04-02T16:12:17.574817+0000","last_undegraded":"2024-04-02T16:12:17.574817+0000","last_fullsized":"2024-04-02T16:12:17.574817+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:02:08.209108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.699846+0000","last_change":"2024-04-02T16:12:14.715075+0000","last_active":"2024-04-02T16:12:16.699846+0000","last_peered":"2024-04-02T16:12:16.699846+0000","last_clean":"2024-04-02T16:12:16.699846+0000","last_became_active":"2024-04-02T16:12:14.714602+0000","last_became_peered":"2024-04-02T16:12:14.714602+0000","last_unstale":"2024-04-02T16:12:16.699846+0000","last_undegraded":"2024-04-02T16:12:16.699846+0000","last_fullsized":"2024-04-02T16:12:16.699846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:10:57.803209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573608+0000","last_change":"2024-04-02T16:12:15.749885+0000","last_active":"2024-04-02T16:12:17.573608+0000","last_peered":"2024-04-02T16:12:17.573608+0000","last_clean":"2024-04-02T16:12:17.573608+0000","last_became_active":"2024-04-02T16:12:15.749230+0000","last_became_peered":"2024-04-02T16:12:15.749230+0000","last_unstale":"2024-04-02T16:12:17.573608+0000","last_undegraded":"2024-04-02T16:12:17.573608+0000","last_fullsized":"2024-04-02T16:12:17.573608+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:47:10.119028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702435+0000","last_change":"2024-04-02T16:12:14.719120+0000","last_active":"2024-04-02T16:12:16.702435+0000","last_peered":"2024-04-02T16:12:16.702435+0000","last_clean":"2024-04-02T16:12:16.702435+0000","last_became_active":"2024-04-02T16:12:14.718359+0000","last_became_peered":"2024-04-02T16:12:14.718359+0000","last_unstale":"2024-04-02T16:12:16.702435+0000","last_undegraded":"2024-04-02T16:12:16.702435+0000","last_fullsized":"2024-04-02T16:12:16.702435+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:20.496833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701174+0000","last_change":"2024-04-02T16:12:15.722412+0000","last_active":"2024-04-02T16:12:16.701174+0000","last_peered":"2024-04-02T16:12:16.701174+0000","last_clean":"2024-04-02T16:12:16.701174+0000","last_became_active":"2024-04-02T16:12:15.721583+0000","last_became_peered":"2024-04-02T16:12:15.721583+0000","last_unstale":"2024-04-02T16:12:16.701174+0000","last_undegraded":"2024-04-02T16:12:16.701174+0000","last_fullsized":"2024-04-02T16:12:16.701174+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:47:10.119028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702070+0000","last_change":"2024-04-02T16:12:14.718750+0000","last_active":"2024-04-02T16:12:16.702070+0000","last_peered":"2024-04-02T16:12:16.702070+0000","last_clean":"2024-04-02T16:12:16.702070+0000","last_became_active":"2024-04-02T16:12:14.718131+0000","last_became_peered":"2024-04-02T16:12:14.718131+0000","last_unstale":"2024-04-02T16:12:16.702070+0000","last_undegraded":"2024-04-02T16:12:16.702070+0000","last_fullsized":"2024-04-02T16:12:16.702070+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:46:07.579680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.700928+0000","last_change":"2024-04-02T16:12:15.747572+0000","last_active":"2024-04-02T16:12:16.700928+0000","last_peered":"2024-04-02T16:12:16.700928+0000","last_clean":"2024-04-02T16:12:16.700928+0000","last_became_active":"2024-04-02T16:12:15.739871+0000","last_became_peered":"2024-04-02T16:12:15.739871+0000","last_unstale":"2024-04-02T16:12:16.700928+0000","last_undegraded":"2024-04-02T16:12:16.700928+0000","last_fullsized":"2024-04-02T16:12:16.700928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:07.415347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575724+0000","last_change":"2024-04-02T16:12:14.712471+0000","last_active":"2024-04-02T16:12:17.575724+0000","last_peered":"2024-04-02T16:12:17.575724+0000","last_clean":"2024-04-02T16:12:17.575724+0000","last_became_active":"2024-04-02T16:12:14.711972+0000","last_became_peered":"2024-04-02T16:12:14.711972+0000","last_unstale":"2024-04-02T16:12:17.575724+0000","last_undegraded":"2024-04-02T16:12:17.575724+0000","last_fullsized":"2024-04-02T16:12:17.575724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573135+0000","last_change":"2024-04-02T16:12:15.737413+0000","last_active":"2024-04-02T16:12:17.573135+0000","last_peered":"2024-04-02T16:12:17.573135+0000","last_clean":"2024-04-02T16:12:17.573135+0000","last_became_active":"2024-04-02T16:12:15.737155+0000","last_became_peered":"2024-04-02T16:12:15.737155+0000","last_unstale":"2024-04-02T16:12:17.573135+0000","last_undegraded":"2024-04-02T16:12:17.573135+0000","last_fullsized":"2024-04-02T16:12:17.573135+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:58:02.033067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.700833+0000","last_change":"2024-04-02T16:12:14.725124+0000","last_active":"2024-04-02T16:12:16.700833+0000","last_peered":"2024-04-02T16:12:16.700833+0000","last_clean":"2024-04-02T16:12:16.700833+0000","last_became_active":"2024-04-02T16:12:14.724472+0000","last_became_peered":"2024-04-02T16:12:14.724472+0000","last_unstale":"2024-04-02T16:12:16.700833+0000","last_undegraded":"2024-04-02T16:12:16.700833+0000","last_fullsized":"2024-04-02T16:12:16.700833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:46.243080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703302+0000","last_change":"2024-04-02T16:12:15.717213+0000","last_active":"2024-04-02T16:12:16.703302+0000","last_peered":"2024-04-02T16:12:16.703302+0000","last_clean":"2024-04-02T16:12:16.703302+0000","last_became_active":"2024-04-02T16:12:15.716394+0000","last_became_peered":"2024-04-02T16:12:15.716394+0000","last_unstale":"2024-04-02T16:12:16.703302+0000","last_undegraded":"2024-04-02T16:12:16.703302+0000","last_fullsized":"2024-04-02T16:12:16.703302+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:07.911973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577669+0000","last_change":"2024-04-02T16:12:14.735619+0000","last_active":"2024-04-02T16:12:17.577669+0000","last_peered":"2024-04-02T16:12:17.577669+0000","last_clean":"2024-04-02T16:12:17.577669+0000","last_became_active":"2024-04-02T16:12:14.735188+0000","last_became_peered":"2024-04-02T16:12:14.735188+0000","last_unstale":"2024-04-02T16:12:17.577669+0000","last_undegraded":"2024-04-02T16:12:17.577669+0000","last_fullsized":"2024-04-02T16:12:17.577669+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:57:26.765132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577421+0000","last_change":"2024-04-02T16:12:15.710399+0000","last_active":"2024-04-02T16:12:17.577421+0000","last_peered":"2024-04-02T16:12:17.577421+0000","last_clean":"2024-04-02T16:12:17.577421+0000","last_became_active":"2024-04-02T16:12:15.710034+0000","last_became_peered":"2024-04-02T16:12:15.710034+0000","last_unstale":"2024-04-02T16:12:17.577421+0000","last_undegraded":"2024-04-02T16:12:17.577421+0000","last_fullsized":"2024-04-02T16:12:17.577421+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:02:38.717234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574263+0000","last_change":"2024-04-02T16:12:14.708255+0000","last_active":"2024-04-02T16:12:17.574263+0000","last_peered":"2024-04-02T16:12:17.574263+0000","last_clean":"2024-04-02T16:12:17.574263+0000","last_became_active":"2024-04-02T16:12:14.707800+0000","last_became_peered":"2024-04-02T16:12:14.707800+0000","last_unstale":"2024-04-02T16:12:17.574263+0000","last_undegraded":"2024-04-02T16:12:17.574263+0000","last_fullsized":"2024-04-02T16:12:17.574263+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576540+0000","last_change":"2024-04-02T16:12:15.719842+0000","last_active":"2024-04-02T16:12:17.576540+0000","last_peered":"2024-04-02T16:12:17.576540+0000","last_clean":"2024-04-02T16:12:17.576540+0000","last_became_active":"2024-04-02T16:12:15.719549+0000","last_became_peered":"2024-04-02T16:12:15.719549+0000","last_unstale":"2024-04-02T16:12:17.576540+0000","last_undegraded":"2024-04-02T16:12:17.576540+0000","last_fullsized":"2024-04-02T16:12:17.576540+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:07.911973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703113+0000","last_change":"2024-04-02T16:12:14.737929+0000","last_active":"2024-04-02T16:12:16.703113+0000","last_peered":"2024-04-02T16:12:16.703113+0000","last_clean":"2024-04-02T16:12:16.703113+0000","last_became_active":"2024-04-02T16:12:14.737178+0000","last_became_peered":"2024-04-02T16:12:14.737178+0000","last_unstale":"2024-04-02T16:12:16.703113+0000","last_undegraded":"2024-04-02T16:12:16.703113+0000","last_fullsized":"2024-04-02T16:12:16.703113+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:58.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702813+0000","last_change":"2024-04-02T16:12:15.722608+0000","last_active":"2024-04-02T16:12:16.702813+0000","last_peered":"2024-04-02T16:12:16.702813+0000","last_clean":"2024-04-02T16:12:16.702813+0000","last_became_active":"2024-04-02T16:12:15.721235+0000","last_became_peered":"2024-04-02T16:12:15.721235+0000","last_unstale":"2024-04-02T16:12:16.702813+0000","last_undegraded":"2024-04-02T16:12:16.702813+0000","last_fullsized":"2024-04-02T16:12:16.702813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:27:42.692500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576694+0000","last_change":"2024-04-02T16:12:14.719497+0000","last_active":"2024-04-02T16:12:17.576694+0000","last_peered":"2024-04-02T16:12:17.576694+0000","last_clean":"2024-04-02T16:12:17.576694+0000","last_became_active":"2024-04-02T16:12:14.719166+0000","last_became_peered":"2024-04-02T16:12:14.719166+0000","last_unstale":"2024-04-02T16:12:17.576694+0000","last_undegraded":"2024-04-02T16:12:17.576694+0000","last_fullsized":"2024-04-02T16:12:17.576694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:26:36.850031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573756+0000","last_change":"2024-04-02T16:12:15.741289+0000","last_active":"2024-04-02T16:12:17.573756+0000","last_peered":"2024-04-02T16:12:17.573756+0000","last_clean":"2024-04-02T16:12:17.573756+0000","last_became_active":"2024-04-02T16:12:15.740988+0000","last_became_peered":"2024-04-02T16:12:15.740988+0000","last_unstale":"2024-04-02T16:12:17.573756+0000","last_undegraded":"2024-04-02T16:12:17.573756+0000","last_fullsized":"2024-04-02T16:12:17.573756+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:09:18.120539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702722+0000","last_change":"2024-04-02T16:12:14.726985+0000","last_active":"2024-04-02T16:12:16.702722+0000","last_peered":"2024-04-02T16:12:16.702722+0000","last_clean":"2024-04-02T16:12:16.702722+0000","last_became_active":"2024-04-02T16:12:14.726281+0000","last_became_peered":"2024-04-02T16:12:14.726281+0000","last_unstale":"2024-04-02T16:12:16.702722+0000","last_undegraded":"2024-04-02T16:12:16.702722+0000","last_fullsized":"2024-04-02T16:12:16.702722+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:21.034455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.577171+0000","last_change":"2024-04-02T16:12:15.715396+0000","last_active":"2024-04-02T16:12:17.577171+0000","last_peered":"2024-04-02T16:12:17.577171+0000","last_clean":"2024-04-02T16:12:17.577171+0000","last_became_active":"2024-04-02T16:12:15.715066+0000","last_became_peered":"2024-04-02T16:12:15.715066+0000","last_unstale":"2024-04-02T16:12:17.577171+0000","last_undegraded":"2024-04-02T16:12:17.577171+0000","last_fullsized":"2024-04-02T16:12:17.577171+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:34:41.912392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573998+0000","last_change":"2024-04-02T16:12:14.724232+0000","last_active":"2024-04-02T16:12:17.573998+0000","last_peered":"2024-04-02T16:12:17.573998+0000","last_clean":"2024-04-02T16:12:17.573998+0000","last_became_active":"2024-04-02T16:12:14.723889+0000","last_became_peered":"2024-04-02T16:12:14.723889+0000","last_unstale":"2024-04-02T16:12:17.573998+0000","last_undegraded":"2024-04-02T16:12:17.573998+0000","last_fullsized":"2024-04-02T16:12:17.573998+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:54:18.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573827+0000","last_change":"2024-04-02T16:12:15.756349+0000","last_active":"2024-04-02T16:12:17.573827+0000","last_peered":"2024-04-02T16:12:17.573827+0000","last_clean":"2024-04-02T16:12:17.573827+0000","last_became_active":"2024-04-02T16:12:15.755718+0000","last_became_peered":"2024-04-02T16:12:15.755718+0000","last_unstale":"2024-04-02T16:12:17.573827+0000","last_undegraded":"2024-04-02T16:12:17.573827+0000","last_fullsized":"2024-04-02T16:12:17.573827+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T04:02:08.209108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575944+0000","last_change":"2024-04-02T16:12:14.735767+0000","last_active":"2024-04-02T16:12:17.575944+0000","last_peered":"2024-04-02T16:12:17.575944+0000","last_clean":"2024-04-02T16:12:17.575944+0000","last_became_active":"2024-04-02T16:12:14.735499+0000","last_became_peered":"2024-04-02T16:12:14.735499+0000","last_unstale":"2024-04-02T16:12:17.575944+0000","last_undegraded":"2024-04-02T16:12:17.575944+0000","last_fullsized":"2024-04-02T16:12:17.575944+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:58.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574686+0000","last_change":"2024-04-02T16:12:15.749729+0000","last_active":"2024-04-02T16:12:17.574686+0000","last_peered":"2024-04-02T16:12:17.574686+0000","last_clean":"2024-04-02T16:12:17.574686+0000","last_became_active":"2024-04-02T16:12:15.749055+0000","last_became_peered":"2024-04-02T16:12:15.749055+0000","last_unstale":"2024-04-02T16:12:17.574686+0000","last_undegraded":"2024-04-02T16:12:17.574686+0000","last_fullsized":"2024-04-02T16:12:17.574686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:07.415347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575838+0000","last_change":"2024-04-02T16:12:14.714641+0000","last_active":"2024-04-02T16:12:17.575838+0000","last_peered":"2024-04-02T16:12:17.575838+0000","last_clean":"2024-04-02T16:12:17.575838+0000","last_became_active":"2024-04-02T16:12:14.714242+0000","last_became_peered":"2024-04-02T16:12:14.714242+0000","last_unstale":"2024-04-02T16:12:17.575838+0000","last_undegraded":"2024-04-02T16:12:17.575838+0000","last_fullsized":"2024-04-02T16:12:17.575838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:50:06.405287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704888+0000","last_change":"2024-04-02T16:12:15.748880+0000","last_active":"2024-04-02T16:12:16.704888+0000","last_peered":"2024-04-02T16:12:16.704888+0000","last_clean":"2024-04-02T16:12:16.704888+0000","last_became_active":"2024-04-02T16:12:15.747825+0000","last_became_peered":"2024-04-02T16:12:15.747825+0000","last_unstale":"2024-04-02T16:12:16.704888+0000","last_undegraded":"2024-04-02T16:12:16.704888+0000","last_fullsized":"2024-04-02T16:12:16.704888+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:31:03.977476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575335+0000","last_change":"2024-04-02T16:12:14.711749+0000","last_active":"2024-04-02T16:12:17.575335+0000","last_peered":"2024-04-02T16:12:17.575335+0000","last_clean":"2024-04-02T16:12:17.575335+0000","last_became_active":"2024-04-02T16:12:14.711308+0000","last_became_peered":"2024-04-02T16:12:14.711308+0000","last_unstale":"2024-04-02T16:12:17.575335+0000","last_undegraded":"2024-04-02T16:12:17.575335+0000","last_fullsized":"2024-04-02T16:12:17.575335+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.703942+0000","last_change":"2024-04-02T16:12:15.720646+0000","last_active":"2024-04-02T16:12:16.703942+0000","last_peered":"2024-04-02T16:12:16.703942+0000","last_clean":"2024-04-02T16:12:16.703942+0000","last_became_active":"2024-04-02T16:12:15.716229+0000","last_became_peered":"2024-04-02T16:12:15.716229+0000","last_unstale":"2024-04-02T16:12:16.703942+0000","last_undegraded":"2024-04-02T16:12:16.703942+0000","last_fullsized":"2024-04-02T16:12:16.703942+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:49:34.800044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704731+0000","last_change":"2024-04-02T16:12:14.728114+0000","last_active":"2024-04-02T16:12:16.704731+0000","last_peered":"2024-04-02T16:12:16.704731+0000","last_clean":"2024-04-02T16:12:16.704731+0000","last_became_active":"2024-04-02T16:12:14.727429+0000","last_became_peered":"2024-04-02T16:12:14.727429+0000","last_unstale":"2024-04-02T16:12:16.704731+0000","last_undegraded":"2024-04-02T16:12:16.704731+0000","last_fullsized":"2024-04-02T16:12:16.704731+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:47.670026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574205+0000","last_change":"2024-04-02T16:12:15.731614+0000","last_active":"2024-04-02T16:12:17.574205+0000","last_peered":"2024-04-02T16:12:17.574205+0000","last_clean":"2024-04-02T16:12:17.574205+0000","last_became_active":"2024-04-02T16:12:15.731255+0000","last_became_peered":"2024-04-02T16:12:15.731255+0000","last_unstale":"2024-04-02T16:12:17.574205+0000","last_undegraded":"2024-04-02T16:12:17.574205+0000","last_fullsized":"2024-04-02T16:12:17.574205+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:20:31.605127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704679+0000","last_change":"2024-04-02T16:12:14.716172+0000","last_active":"2024-04-02T16:12:16.704679+0000","last_peered":"2024-04-02T16:12:16.704679+0000","last_clean":"2024-04-02T16:12:16.704679+0000","last_became_active":"2024-04-02T16:12:14.714967+0000","last_became_peered":"2024-04-02T16:12:14.714967+0000","last_unstale":"2024-04-02T16:12:16.704679+0000","last_undegraded":"2024-04-02T16:12:16.704679+0000","last_fullsized":"2024-04-02T16:12:16.704679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:38.889079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.700058+0000","last_change":"2024-04-02T16:12:15.731379+0000","last_active":"2024-04-02T16:12:16.700058+0000","last_peered":"2024-04-02T16:12:16.700058+0000","last_clean":"2024-04-02T16:12:16.700058+0000","last_became_active":"2024-04-02T16:12:15.731066+0000","last_became_peered":"2024-04-02T16:12:15.731066+0000","last_unstale":"2024-04-02T16:12:16.700058+0000","last_undegraded":"2024-04-02T16:12:16.700058+0000","last_fullsized":"2024-04-02T16:12:16.700058+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:12:22.044515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704130+0000","last_change":"2024-04-02T16:12:14.718370+0000","last_active":"2024-04-02T16:12:16.704130+0000","last_peered":"2024-04-02T16:12:16.704130+0000","last_clean":"2024-04-02T16:12:16.704130+0000","last_became_active":"2024-04-02T16:12:14.717749+0000","last_became_peered":"2024-04-02T16:12:14.717749+0000","last_unstale":"2024-04-02T16:12:16.704130+0000","last_undegraded":"2024-04-02T16:12:16.704130+0000","last_fullsized":"2024-04-02T16:12:16.704130+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:09:33.433297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.698769+0000","last_change":"2024-04-02T16:12:15.728654+0000","last_active":"2024-04-02T16:12:16.698769+0000","last_peered":"2024-04-02T16:12:16.698769+0000","last_clean":"2024-04-02T16:12:16.698769+0000","last_became_active":"2024-04-02T16:12:15.728359+0000","last_became_peered":"2024-04-02T16:12:15.728359+0000","last_unstale":"2024-04-02T16:12:16.698769+0000","last_undegraded":"2024-04-02T16:12:16.698769+0000","last_fullsized":"2024-04-02T16:12:16.698769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:52:47.253139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573384+0000","last_change":"2024-04-02T16:12:14.717818+0000","last_active":"2024-04-02T16:12:17.573384+0000","last_peered":"2024-04-02T16:12:17.573384+0000","last_clean":"2024-04-02T16:12:17.573384+0000","last_became_active":"2024-04-02T16:12:14.717461+0000","last_became_peered":"2024-04-02T16:12:14.717461+0000","last_unstale":"2024-04-02T16:12:17.573384+0000","last_undegraded":"2024-04-02T16:12:17.573384+0000","last_fullsized":"2024-04-02T16:12:17.573384+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:38:34.580931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.699637+0000","last_change":"2024-04-02T16:12:15.714760+0000","last_active":"2024-04-02T16:12:16.699637+0000","last_peered":"2024-04-02T16:12:16.699637+0000","last_clean":"2024-04-02T16:12:16.699637+0000","last_became_active":"2024-04-02T16:12:15.714170+0000","last_became_peered":"2024-04-02T16:12:15.714170+0000","last_unstale":"2024-04-02T16:12:16.699637+0000","last_undegraded":"2024-04-02T16:12:16.699637+0000","last_fullsized":"2024-04-02T16:12:16.699637+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:02:21.506893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575047+0000","last_change":"2024-04-02T16:12:14.729387+0000","last_active":"2024-04-02T16:12:17.575047+0000","last_peered":"2024-04-02T16:12:17.575047+0000","last_clean":"2024-04-02T16:12:17.575047+0000","last_became_active":"2024-04-02T16:12:14.728744+0000","last_became_peered":"2024-04-02T16:12:14.728744+0000","last_unstale":"2024-04-02T16:12:17.575047+0000","last_undegraded":"2024-04-02T16:12:17.575047+0000","last_fullsized":"2024-04-02T16:12:17.575047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:15:24.931785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.698742+0000","last_change":"2024-04-02T16:12:15.713578+0000","last_active":"2024-04-02T16:12:16.698742+0000","last_peered":"2024-04-02T16:12:16.698742+0000","last_clean":"2024-04-02T16:12:16.698742+0000","last_became_active":"2024-04-02T16:12:15.713313+0000","last_became_peered":"2024-04-02T16:12:15.713313+0000","last_unstale":"2024-04-02T16:12:16.698742+0000","last_undegraded":"2024-04-02T16:12:16.698742+0000","last_fullsized":"2024-04-02T16:12:16.698742+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:54:55.546610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702196+0000","last_change":"2024-04-02T16:12:14.728831+0000","last_active":"2024-04-02T16:12:16.702196+0000","last_peered":"2024-04-02T16:12:16.702196+0000","last_clean":"2024-04-02T16:12:16.702196+0000","last_became_active":"2024-04-02T16:12:14.727741+0000","last_became_peered":"2024-04-02T16:12:14.727741+0000","last_unstale":"2024-04-02T16:12:16.702196+0000","last_undegraded":"2024-04-02T16:12:16.702196+0000","last_fullsized":"2024-04-02T16:12:16.702196+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:05:19.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574646+0000","last_change":"2024-04-02T16:12:15.739569+0000","last_active":"2024-04-02T16:12:17.574646+0000","last_peered":"2024-04-02T16:12:17.574646+0000","last_clean":"2024-04-02T16:12:17.574646+0000","last_became_active":"2024-04-02T16:12:15.738815+0000","last_became_peered":"2024-04-02T16:12:15.738815+0000","last_unstale":"2024-04-02T16:12:17.574646+0000","last_undegraded":"2024-04-02T16:12:17.574646+0000","last_fullsized":"2024-04-02T16:12:17.574646+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:07.911973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.701237+0000","last_change":"2024-04-02T16:12:14.737907+0000","last_active":"2024-04-02T16:12:16.701237+0000","last_peered":"2024-04-02T16:12:16.701237+0000","last_clean":"2024-04-02T16:12:16.701237+0000","last_became_active":"2024-04-02T16:12:14.737140+0000","last_became_peered":"2024-04-02T16:12:14.737140+0000","last_unstale":"2024-04-02T16:12:16.701237+0000","last_undegraded":"2024-04-02T16:12:16.701237+0000","last_fullsized":"2024-04-02T16:12:16.701237+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:57:26.765132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.700045+0000","last_change":"2024-04-02T16:12:15.742007+0000","last_active":"2024-04-02T16:12:16.700045+0000","last_peered":"2024-04-02T16:12:16.700045+0000","last_clean":"2024-04-02T16:12:16.700045+0000","last_became_active":"2024-04-02T16:12:15.741685+0000","last_became_peered":"2024-04-02T16:12:15.741685+0000","last_unstale":"2024-04-02T16:12:16.700045+0000","last_undegraded":"2024-04-02T16:12:16.700045+0000","last_fullsized":"2024-04-02T16:12:16.700045+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:53:48.680086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575323+0000","last_change":"2024-04-02T16:12:14.725690+0000","last_active":"2024-04-02T16:12:17.575323+0000","last_peered":"2024-04-02T16:12:17.575323+0000","last_clean":"2024-04-02T16:12:17.575323+0000","last_became_active":"2024-04-02T16:12:14.725425+0000","last_became_peered":"2024-04-02T16:12:14.725425+0000","last_unstale":"2024-04-02T16:12:17.575323+0000","last_undegraded":"2024-04-02T16:12:17.575323+0000","last_fullsized":"2024-04-02T16:12:17.575323+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:08.157373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.704256+0000","last_change":"2024-04-02T16:12:15.735774+0000","last_active":"2024-04-02T16:12:16.704256+0000","last_peered":"2024-04-02T16:12:16.704256+0000","last_clean":"2024-04-02T16:12:16.704256+0000","last_became_active":"2024-04-02T16:12:15.727721+0000","last_became_peered":"2024-04-02T16:12:15.727721+0000","last_unstale":"2024-04-02T16:12:16.704256+0000","last_undegraded":"2024-04-02T16:12:16.704256+0000","last_fullsized":"2024-04-02T16:12:16.704256+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:37:13.746991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574198+0000","last_change":"2024-04-02T16:12:14.711273+0000","last_active":"2024-04-02T16:12:17.574198+0000","last_peered":"2024-04-02T16:12:17.574198+0000","last_clean":"2024-04-02T16:12:17.574198+0000","last_became_active":"2024-04-02T16:12:14.710880+0000","last_became_peered":"2024-04-02T16:12:14.710880+0000","last_unstale":"2024-04-02T16:12:17.574198+0000","last_undegraded":"2024-04-02T16:12:17.574198+0000","last_fullsized":"2024-04-02T16:12:17.574198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573158+0000","last_change":"2024-04-02T16:12:15.710718+0000","last_active":"2024-04-02T16:12:17.573158+0000","last_peered":"2024-04-02T16:12:17.573158+0000","last_clean":"2024-04-02T16:12:17.573158+0000","last_became_active":"2024-04-02T16:12:15.710393+0000","last_became_peered":"2024-04-02T16:12:15.710393+0000","last_unstale":"2024-04-02T16:12:17.573158+0000","last_undegraded":"2024-04-02T16:12:17.573158+0000","last_fullsized":"2024-04-02T16:12:17.573158+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:48:51.548990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574534+0000","last_change":"2024-04-02T16:12:14.716204+0000","last_active":"2024-04-02T16:12:17.574534+0000","last_peered":"2024-04-02T16:12:17.574534+0000","last_clean":"2024-04-02T16:12:17.574534+0000","last_became_active":"2024-04-02T16:12:14.715693+0000","last_became_peered":"2024-04-02T16:12:14.715693+0000","last_unstale":"2024-04-02T16:12:17.574534+0000","last_undegraded":"2024-04-02T16:12:17.574534+0000","last_fullsized":"2024-04-02T16:12:17.574534+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:58.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574848+0000","last_change":"2024-04-02T16:12:15.729489+0000","last_active":"2024-04-02T16:12:17.574848+0000","last_peered":"2024-04-02T16:12:17.574848+0000","last_clean":"2024-04-02T16:12:17.574848+0000","last_became_active":"2024-04-02T16:12:15.728770+0000","last_became_peered":"2024-04-02T16:12:15.728770+0000","last_unstale":"2024-04-02T16:12:17.574848+0000","last_undegraded":"2024-04-02T16:12:17.574848+0000","last_fullsized":"2024-04-02T16:12:17.574848+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:34:26.572656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573631+0000","last_change":"2024-04-02T16:12:14.723590+0000","last_active":"2024-04-02T16:12:17.573631+0000","last_peered":"2024-04-02T16:12:17.573631+0000","last_clean":"2024-04-02T16:12:17.573631+0000","last_became_active":"2024-04-02T16:12:14.723145+0000","last_became_peered":"2024-04-02T16:12:14.723145+0000","last_unstale":"2024-04-02T16:12:17.573631+0000","last_undegraded":"2024-04-02T16:12:17.573631+0000","last_fullsized":"2024-04-02T16:12:17.573631+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:55:58.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.573333+0000","last_change":"2024-04-02T16:12:15.723737+0000","last_active":"2024-04-02T16:12:17.573333+0000","last_peered":"2024-04-02T16:12:17.573333+0000","last_clean":"2024-04-02T16:12:17.573333+0000","last_became_active":"2024-04-02T16:12:15.721696+0000","last_became_peered":"2024-04-02T16:12:15.721696+0000","last_unstale":"2024-04-02T16:12:17.573333+0000","last_undegraded":"2024-04-02T16:12:17.573333+0000","last_fullsized":"2024-04-02T16:12:17.573333+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:51:32.088733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574247+0000","last_change":"2024-04-02T16:12:14.725705+0000","last_active":"2024-04-02T16:12:17.574247+0000","last_peered":"2024-04-02T16:12:17.574247+0000","last_clean":"2024-04-02T16:12:17.574247+0000","last_became_active":"2024-04-02T16:12:14.725450+0000","last_became_peered":"2024-04-02T16:12:14.725450+0000","last_unstale":"2024-04-02T16:12:17.574247+0000","last_undegraded":"2024-04-02T16:12:17.574247+0000","last_fullsized":"2024-04-02T16:12:17.574247+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T02:37:08.145090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.575790+0000","last_change":"2024-04-02T16:12:15.712028+0000","last_active":"2024-04-02T16:12:17.575790+0000","last_peered":"2024-04-02T16:12:17.575790+0000","last_clean":"2024-04-02T16:12:17.575790+0000","last_became_active":"2024-04-02T16:12:15.711723+0000","last_became_peered":"2024-04-02T16:12:15.711723+0000","last_unstale":"2024-04-02T16:12:17.575790+0000","last_undegraded":"2024-04-02T16:12:17.575790+0000","last_fullsized":"2024-04-02T16:12:17.575790+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:27.972798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.702453+0000","last_change":"2024-04-02T16:12:14.726690+0000","last_active":"2024-04-02T16:12:16.702453+0000","last_peered":"2024-04-02T16:12:16.702453+0000","last_clean":"2024-04-02T16:12:16.702453+0000","last_became_active":"2024-04-02T16:12:14.725905+0000","last_became_peered":"2024-04-02T16:12:14.725905+0000","last_unstale":"2024-04-02T16:12:16.702453+0000","last_undegraded":"2024-04-02T16:12:16.702453+0000","last_fullsized":"2024-04-02T16:12:16.702453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:37:38.639810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:16.700886+0000","last_change":"2024-04-02T16:12:15.747285+0000","last_active":"2024-04-02T16:12:16.700886+0000","last_peered":"2024-04-02T16:12:16.700886+0000","last_clean":"2024-04-02T16:12:16.700886+0000","last_became_active":"2024-04-02T16:12:15.739227+0000","last_became_peered":"2024-04-02T16:12:15.739227+0000","last_unstale":"2024-04-02T16:12:16.700886+0000","last_undegraded":"2024-04-02T16:12:16.700886+0000","last_fullsized":"2024-04-02T16:12:16.700886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:26:37.860090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.576048+0000","last_change":"2024-04-02T16:12:14.736388+0000","last_active":"2024-04-02T16:12:17.576048+0000","last_peered":"2024-04-02T16:12:17.576048+0000","last_clean":"2024-04-02T16:12:17.576048+0000","last_became_active":"2024-04-02T16:12:14.736096+0000","last_became_peered":"2024-04-02T16:12:14.736096+0000","last_unstale":"2024-04-02T16:12:17.576048+0000","last_undegraded":"2024-04-02T16:12:17.576048+0000","last_fullsized":"2024-04-02T16:12:17.576048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T03:34:25.562596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574049+0000","last_change":"2024-04-02T16:12:15.747897+0000","last_active":"2024-04-02T16:12:17.574049+0000","last_peered":"2024-04-02T16:12:17.574049+0000","last_clean":"2024-04-02T16:12:17.574049+0000","last_became_active":"2024-04-02T16:12:15.741031+0000","last_became_peered":"2024-04-02T16:12:15.741031+0000","last_unstale":"2024-04-02T16:12:17.574049+0000","last_undegraded":"2024-04-02T16:12:17.574049+0000","last_fullsized":"2024-04-02T16:12:17.574049+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:14.679647+0000","last_clean_scrub_stamp":"2024-04-02T16:12:14.679647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:59:27.972798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574044+0000","last_change":"2024-04-02T16:12:14.735597+0000","last_active":"2024-04-02T16:12:17.574044+0000","last_peered":"2024-04-02T16:12:17.574044+0000","last_clean":"2024-04-02T16:12:17.574044+0000","last_became_active":"2024-04-02T16:12:14.734832+0000","last_became_peered":"2024-04-02T16:12:14.734832+0000","last_unstale":"2024-04-02T16:12:17.574044+0000","last_undegraded":"2024-04-02T16:12:17.574044+0000","last_fullsized":"2024-04-02T16:12:17.574044+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:12:13.669587+0000","last_clean_scrub_stamp":"2024-04-02T16:12:13.669587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:50:11.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-02T16:12:17.574069+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:17.574069+0000","last_peered":"2024-04-02T16:12:17.574069+0000","last_clean":"2024-04-02T16:12:17.574069+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:17.574069+0000","last_undegraded":"2024-04-02T16:12:17.574069+0000","last_fullsized":"2024-04-02T16:12:17.574069+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:23.611 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T16:12:23.611 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-02T16:12:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 Processor -- start 2024-04-02T16:12:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 -- start start 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d20073230 0x7f3d200b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d200b19e0 con 0x7f3d20148e50 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d200b05c0 con 0x7f3d200b23d0 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d27f1e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3d200b08a0 con 0x7f3d20073230 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d264bb640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3d20148e50 0x7f3d20149280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55952/0 (socket says 172.21.15.40:55952) 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d264bb640 1 -- 172.21.15.40:0/1519225614 learned_addr learned my addr 172.21.15.40:0/1519225614 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d254b9640 1 --2- 172.21.15.40:0/1519225614 >> v2:172.21.15.40:3300/0 conn(0x7f3d20073230 0x7f3d200b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d24cb8640 1 -- 172.21.15.40:0/1519225614 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424853731 0 0) 0x7f3d200b19e0 con 0x7f3d20148e50 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d24cb8640 1 -- 172.21.15.40:0/1519225614 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d0c003530 con 0x7f3d20148e50 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d254b9640 1 -- 172.21.15.40:0/1519225614 >> v1:172.21.15.204:6789/0 conn(0x7f3d200b23d0 legacy=0x7f3d2009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d254b9640 1 -- 172.21.15.40:0/1519225614 >> v1:172.21.15.40:6789/0 conn(0x7f3d20148e50 legacy=0x7f3d20149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.784+0000 7f3d254b9640 1 -- 172.21.15.40:0/1519225614 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d200b0b80 con 0x7f3d20073230 2024-04-02T16:12:23.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d254b9640 1 --2- 172.21.15.40:0/1519225614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d20073230 0x7f3d200b27b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097c0 tx=0x7f3d1c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca62ad78ff8bfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d24cb8640 1 -- 172.21.15.40:0/1519225614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0b0070 con 0x7f3d20073230 2024-04-02T16:12:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d24cb8640 1 -- 172.21.15.40:0/1519225614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1c0ab600 con 0x7f3d20073230 2024-04-02T16:12:23.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d24cb8640 1 -- 172.21.15.40:0/1519225614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0ab9d0 con 0x7f3d20073230 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 -- 172.21.15.40:0/1519225614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d20073230 msgr2=0x7f3d200b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/1519225614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d20073230 0x7f3d200b27b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097c0 tx=0x7f3d1c01c410 comp rx=0 tx=0).stop 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 -- 172.21.15.40:0/1519225614 shutdown_connections 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/1519225614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d20073230 0x7f3d200b27b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 -- 172.21.15.40:0/1519225614 >> 172.21.15.40:0/1519225614 conn(0x7f3d200b8fd0 msgr2=0x7f3d200b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 -- 172.21.15.40:0/1519225614 shutdown_connections 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.788+0000 7f3d27f1e640 1 -- 172.21.15.40:0/1519225614 wait complete. 2024-04-02T16:12:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 Processor -- start 2024-04-02T16:12:23.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- start start 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d20073230 0x7f3d20141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d20148e50 0x7f3d201405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d20153a30 con 0x7f3d200b23d0 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d201538b0 con 0x7f3d20148e50 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d20153bb0 con 0x7f3d20073230 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59356/0 (socket says 172.21.15.40:59356) 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 -- 172.21.15.40:0/2189972148 learned_addr learned my addr 172.21.15.40:0/2189972148 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d25cba640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d20073230 0x7f3d20141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d264bb640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d20148e50 0x7f3d201405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 -- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d20073230 msgr2=0x7f3d20141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d20073230 0x7f3d20141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 -- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d20148e50 msgr2=0x7f3d201405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d20148e50 0x7f3d201405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 -- 172.21.15.40:0/2189972148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d20141920 con 0x7f3d200b23d0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d254b9640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ae040 tx=0x7f3d1c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0b8020 con 0x7f3d200b23d0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1c0b4410 con 0x7f3d200b23d0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0b0040 con 0x7f3d200b23d0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d201429d0 con 0x7f3d200b23d0 2024-04-02T16:12:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d20140b00 con 0x7f3d200b23d0 2024-04-02T16:12:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2013fec0 con 0x7f3d200b23d0 2024-04-02T16:12:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.792+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d1c0bb020 con 0x7f3d200b23d0 2024-04-02T16:12:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.796+0000 7f3d127fc640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 0x7f3d0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.796+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3d1c1204d0 con 0x7f3d200b23d0 2024-04-02T16:12:23.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.796+0000 7f3d25cba640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 0x7f3d0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:23.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.796+0000 7f3d25cba640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 0x7f3d0406a840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001650 tx=0x7f3d14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:23.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:23.800+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3d2013fec0 con 0x7f3d200b23d0 2024-04-02T16:12:24.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.024+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3d200685f0 con 0x7f3d200b23d0 2024-04-02T16:12:24.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d127fc640 1 -- 172.21.15.40:0/2189972148 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f3d1c11ee20 con 0x7f3d200b23d0 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD5LQxm3/mnOBAA99WD/i8maqhym9P/RBkKdw== 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-02T16:12:24.028 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2024-04-02T16:12:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 msgr2=0x7f3d0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 0x7f3d0406a840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d14001650 tx=0x7f3d14009040 comp rx=0 tx=0).stop 2024-04-02T16:12:24.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 msgr2=0x7f3d2013f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ae040 tx=0x7f3d1c0191b0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 shutdown_connections 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3d04068380 0x7f3d0406a840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d20148e50 0x7f3d201405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d200b23d0 0x7f3d2013f4e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 --2- 172.21.15.40:0/2189972148 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d20073230 0x7f3d20141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 >> 172.21.15.40:0/2189972148 conn(0x7f3d200b8fd0 msgr2=0x7f3d2006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 shutdown_connections 2024-04-02T16:12:24.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.028+0000 7f3d27f1e640 1 -- 172.21.15.40:0/2189972148 wait complete. 2024-04-02T16:12:24.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.172+0000 7f6bb029e640 1 Processor -- start 2024-04-02T16:12:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.172+0000 7f6bb029e640 1 -- start start 2024-04-02T16:12:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b23d0 0x7f6ba80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ba8073230 0x7f6ba809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba809dfe0 0x7f6ba8148e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80b19e0 con 0x7f6ba80b23d0 2024-04-02T16:12:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80b05c0 con 0x7f6ba8073230 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bb029e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba80b08a0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b23d0 0x7f6ba80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b23d0 0x7f6ba80b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59372/0 (socket says 172.21.15.40:59372) 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 -- 172.21.15.40:0/1204435539 learned_addr learned my addr 172.21.15.40:0/1204435539 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bad839640 1 --2- 172.21.15.40:0/1204435539 >> v2:172.21.15.204:3300/0 conn(0x7f6ba8073230 0x7f6ba809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae83b640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba809dfe0 0x7f6ba8148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 -- 172.21.15.40:0/1204435539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba809dfe0 msgr2=0x7f6ba8148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba809dfe0 0x7f6ba8148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 -- 172.21.15.40:0/1204435539 >> v2:172.21.15.204:3300/0 conn(0x7f6ba8073230 msgr2=0x7f6ba809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1204435539 >> v2:172.21.15.204:3300/0 conn(0x7f6ba8073230 0x7f6ba809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 -- 172.21.15.40:0/1204435539 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80b0b80 con 0x7f6ba80b23d0 2024-04-02T16:12:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b23d0 0x7f6ba80b27b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8148b00 tx=0x7f6ba401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c749f6a274a5599 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bad038640 1 -- 172.21.15.40:0/1204435539 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40b0070 con 0x7f6ba80b23d0 2024-04-02T16:12:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bad038640 1 -- 172.21.15.40:0/1204435539 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba40ab600 con 0x7f6ba80b23d0 2024-04-02T16:12:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.176+0000 7f6bad038640 1 -- 172.21.15.40:0/1204435539 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40ab9d0 con 0x7f6ba80b23d0 2024-04-02T16:12:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1204435539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b23d0 msgr2=0x7f6ba80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b23d0 0x7f6ba80b27b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8148b00 tx=0x7f6ba401c490 comp rx=0 tx=0).stop 2024-04-02T16:12:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1204435539 shutdown_connections 2024-04-02T16:12:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba809dfe0 0x7f6ba8148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1204435539 >> v2:172.21.15.204:3300/0 conn(0x7f6ba8073230 0x7f6ba809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1204435539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b23d0 0x7f6ba80b27b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1204435539 >> 172.21.15.40:0/1204435539 conn(0x7f6ba80b8fd0 msgr2=0x7f6ba80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1204435539 shutdown_connections 2024-04-02T16:12:24.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1204435539 wait complete. 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 Processor -- start 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- start start 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8073230 0x7f6ba81e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba80b23d0 0x7f6ba81e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81460f0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8145f70 con 0x7f6ba8073230 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba8146270 con 0x7f6ba80b23d0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59384/0 (socket says 172.21.15.40:59384) 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 -- 172.21.15.40:0/1522126984 learned_addr learned my addr 172.21.15.40:0/1522126984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8073230 0x7f6ba81e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bae83b640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba80b23d0 0x7f6ba81e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 -- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba80b23d0 msgr2=0x7f6ba81e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba80b23d0 0x7f6ba81e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 -- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8073230 msgr2=0x7f6ba81e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8073230 0x7f6ba81e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 -- 172.21.15.40:0/1522126984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba809acc0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bad839640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800d5f0 tx=0x7f6b9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980125f0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98019070 con 0x7f6ba809dfe0 2024-04-02T16:12:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba8071940 con 0x7f6ba809dfe0 2024-04-02T16:12:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980129b0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.180+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba80722e0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba80685f0 con 0x7f6ba809dfe0 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f6b98012b50 con 0x7f6ba809dfe0 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6b9effd640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 0x7f6b8c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f6b98086c30 con 0x7f6ba809dfe0 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 0x7f6b8c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.184+0000 7f6bae03a640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 0x7f6b8c06a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40ad000 tx=0x7f6ba4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.188+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f6b98056c30 con 0x7f6ba809dfe0 2024-04-02T16:12:24.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.388+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ba80b1690 con 0x7f6ba809dfe0 2024-04-02T16:12:24.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6b9effd640 1 -- 172.21.15.40:0/1522126984 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7f6b9805af30 con 0x7f6ba809dfe0 2024-04-02T16:12:24.393 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:24.393 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:24.393 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 msgr2=0x7f6b8c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 0x7f6b8c06a840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40ad000 tx=0x7f6ba4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 msgr2=0x7f6ba806feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800d5f0 tx=0x7f6b9800dab0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 shutdown_connections 2024-04-02T16:12:24.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ba80b23d0 0x7f6ba81e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6b8c068380 0x7f6b8c06a840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba809dfe0 0x7f6ba806feb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 --2- 172.21.15.40:0/1522126984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ba8073230 0x7f6ba81e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 >> 172.21.15.40:0/1522126984 conn(0x7f6ba80b8fd0 msgr2=0x7f6ba806cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 shutdown_connections 2024-04-02T16:12:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.392+0000 7f6bb029e640 1 -- 172.21.15.40:0/1522126984 wait complete. 2024-04-02T16:12:24.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T16:12:15.068670+0000', 'modified': '2024-04-02T16:12:17.070715+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4989}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4989': {'gid': 4989, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/2966573419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 2966573419}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 2966573419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4989, 'qdb_cluster': [4989]} 2024-04-02T16:12:24.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T16:12:24.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-02T16:12:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 Processor -- start 2024-04-02T16:12:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 -- start start 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea880ae080 0x7fea880b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b1c10 0x7fea880b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea880b3210 con 0x7fea880b25b0 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fea8806db10 con 0x7fea880ae080 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea8dbad640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea880ae460 con 0x7fea880b1c10 2024-04-02T16:12:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea877fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea880ae080 0x7fea880b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea87fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fea880b25b0 0x7fea8806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55962/0 (socket says 172.21.15.40:55962) 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea87fff640 1 -- 172.21.15.40:0/763648500 learned_addr learned my addr 172.21.15.40:0/763648500 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea86ffd640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b1c10 0x7fea880b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea877fe640 1 -- 172.21.15.40:0/763648500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b1c10 msgr2=0x7fea880b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea877fe640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b1c10 0x7fea880b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea877fe640 1 -- 172.21.15.40:0/763648500 >> v1:172.21.15.40:6789/0 conn(0x7fea880b25b0 legacy=0x7fea8806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.748+0000 7fea877fe640 1 -- 172.21.15.40:0/763648500 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea880af670 con 0x7fea880ae080 2024-04-02T16:12:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea877fe640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880ae080 0x7fea880b0300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4241bfbacc522e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea867fc640 1 -- 172.21.15.40:0/763648500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b0070 con 0x7fea880ae080 2024-04-02T16:12:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea867fc640 1 -- 172.21.15.40:0/763648500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800a4e50 con 0x7fea880ae080 2024-04-02T16:12:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea867fc640 1 -- 172.21.15.40:0/763648500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800ab820 con 0x7fea880ae080 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- 172.21.15.40:0/763648500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880ae080 msgr2=0x7fea880b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880ae080 0x7fea880b0300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fea800097c0 tx=0x7fea8001c4a0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- 172.21.15.40:0/763648500 shutdown_connections 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b1c10 0x7fea880b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- 172.21.15.40:0/763648500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880ae080 0x7fea880b0300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- 172.21.15.40:0/763648500 >> 172.21.15.40:0/763648500 conn(0x7fea880b8fd0 msgr2=0x7fea880b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- 172.21.15.40:0/763648500 shutdown_connections 2024-04-02T16:12:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- 172.21.15.40:0/763648500 wait complete. 2024-04-02T16:12:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 Processor -- start 2024-04-02T16:12:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- start start 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea880ae080 0x7fea88150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 0x7fea88151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea880aeeb0 con 0x7fea880b1c10 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea880af2f0 con 0x7fea880ae080 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea8dbad640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea880af030 con 0x7fea880b25b0 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59414/0 (socket says 172.21.15.40:59414) 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 -- 172.21.15.40:0/3333268295 learned_addr learned my addr 172.21.15.40:0/3333268295 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea87fff640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 0x7fea88151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea877fe640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea880ae080 0x7fea88150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 -- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 msgr2=0x7fea88151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 0x7fea88151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 -- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea880ae080 msgr2=0x7fea88150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea880ae080 0x7fea88150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 -- 172.21.15.40:0/3333268295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea88151f20 con 0x7fea880b1c10 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea87fff640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 0x7fea88151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.752+0000 7fea86ffd640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea70007ef0 tx=0x7fea7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 -- 172.21.15.40:0/3333268295 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70014620 con 0x7fea880b1c10 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 -- 172.21.15.40:0/3333268295 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea70019070 con 0x7fea880b1c10 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8809ccf0 con 0x7fea880b1c10 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 -- 172.21.15.40:0/3333268295 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70014a00 con 0x7fea880b1c10 2024-04-02T16:12:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881502c0 con 0x7fea880b1c10 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 -- 172.21.15.40:0/3333268295 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fea70014ba0 con 0x7fea880b1c10 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 0x7fea6806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea7ffff640 1 -- 172.21.15.40:0/3333268295 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fea700872f0 con 0x7fea880b1c10 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea877fe640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 0x7fea6806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 Processor -- start 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 -- start start 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea877fe640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 0x7fea6806a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea800ad000 tx=0x7fea80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f153850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef72980 con 0x55b02f155ca0 2024-04-02T16:12:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef0fbd0 con 0x55b02f12acd0 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.756+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55b02ef6e930 con 0x55b02f155380 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59424/0 (socket says 172.21.15.40:59424) 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/365700509 learned_addr learned my addr 172.21.15.40:0/365700509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea877fe640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea86ffd640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 msgr2=0x55b02f155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 msgr2=0x55b02f153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/365700509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02ef4cc30 con 0x55b02f155ca0 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea86ffd640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fea78001720 tx=0x7fea78001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67a0f63152ad392 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea7d7fa640 1 -- 172.21.15.40:0/365700509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78018070 con 0x55b02f155ca0 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea7d7fa640 1 -- 172.21.15.40:0/365700509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea78005cd0 con 0x55b02f155ca0 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea7d7fa640 1 -- 172.21.15.40:0/365700509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780072f0 con 0x55b02f155ca0 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/365700509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 msgr2=0x55b02f160610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fea78001720 tx=0x7fea78001be0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/365700509 shutdown_connections 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f160610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- 172.21.15.40:0/365700509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/365700509 >> 172.21.15.40:0/365700509 conn(0x55b02ec36780 msgr2=0x55b02f153140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/365700509 shutdown_connections 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/365700509 wait complete. 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- start start 2024-04-02T16:12:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 Processor -- start 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f1260d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f126610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef30530 con 0x55b02f155ca0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef4d7f0 con 0x55b02f12acd0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55b02ef7f090 con 0x55b02f155380 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 10 client.0 _ll_register_callbacks cb 0x55b02efd4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59426/0 (socket says 172.21.15.40:59426) 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/2521633996 learned_addr learned my addr 172.21.15.40:0/2521633996 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea877fe640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f1260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea86ffd640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f126610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 msgr2=0x55b02f126610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f126610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 msgr2=0x55b02f1260d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f1260d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 -- 172.21.15.40:0/2521633996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02efb8750 con 0x55b02f155ca0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea86ffd640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea877fe640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f1260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea87fff640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea78016040 tx=0x7fea78000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5575c153f62b7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea667fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78023020 con 0x55b02f155ca0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fea78005ad0 con 0x55b02f155ca0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78005cd0 con 0x55b02f155ca0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fea7802d050 con 0x55b02f155ca0 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 10 client.5184 fetch_fsmap learned FSMap version 14 2024-04-02T16:12:24.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.760+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55b02ef0a7b0 con 0x55b02f155ca0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea78005cd0 con 0x55b02f155ca0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 2284+0+0 (secure 0 0 0) 0x7fea78030020 con 0x55b02f155ca0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 10 client.5184 fetch_fsmap finished waiting for FSMap version 14 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 10 client.5184 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 20 client.5184 populate_metadata read hostname 'smithi040' 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f168920 0x55b02f16ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 4 client.5184 mds_command: new command op to 4989 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 --> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02ef0dbc0 con 0x55b02f168920 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea877fe640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f168920 0x55b02f16ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea877fe640 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f168920 0x55b02f16ad10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfc1be68d296150 server_cookie=cc12dabca36511ce in_seq=0 out_seq=0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 10 client.5184 ms_handle_connect on v2:172.21.15.40:6832/2966573419 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 1 -- 172.21.15.40:0/2521633996 <== mds.0 v2:172.21.15.40:6832/2966573419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b02ef0dbc0 con 0x55b02f168920 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 10 client.5184 handle_command_reply: tid=0 2024-04-02T16:12:24.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 client.5184 shutdown 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 msgr2=0x55b02f126b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fea78016040 tx=0x7fea78000ee0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 shutdown_connections 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f168920 0x55b02f16ad10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155380 0x55b02f126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f155ca0 0x55b02f126b50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea667fc640 0 client.5184 ms_handle_reset on v2:172.21.15.40:6832/2966573419 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 --2- 172.21.15.40:0/2521633996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f12acd0 0x55b02f1260d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 >> 172.21.15.40:0/2521633996 conn(0x55b02ec36780 msgr2=0x55b02f153140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 shutdown_connections 2024-04-02T16:12:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 1 -- 172.21.15.40:0/2521633996 wait complete. 2024-04-02T16:12:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.764+0000 7fea90d2c000 20 client.5184 trim_cache size 0 max 16384 2024-04-02T16:12:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.780+0000 7fea90d2c000 1 Processor -- start 2024-04-02T16:12:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.780+0000 7fea90d2c000 1 -- start start 2024-04-02T16:12:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f154100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ecc30 con 0x55b02f12acd0 2024-04-02T16:12:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ebd90 con 0x55b02f155380 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55b02ef71c90 con 0x55b02f155ca0 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59428/0 (socket says 172.21.15.40:59428) 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3589410044 learned_addr learned my addr 172.21.15.40:0/3589410044 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 msgr2=0x55b02f154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 msgr2=0x55b02f16e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3589410044 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02eefae80 con 0x55b02f12acd0 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fea800a4fd0 tx=0x7fea800a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4dd36677af5c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea7cff9640 1 -- 172.21.15.40:0/3589410044 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b0040 con 0x55b02f12acd0 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea7cff9640 1 -- 172.21.15.40:0/3589410044 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800c72d0 con 0x55b02f12acd0 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea7cff9640 1 -- 172.21.15.40:0/3589410044 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800c7680 con 0x55b02f12acd0 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- 172.21.15.40:0/3589410044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 msgr2=0x55b02f16dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fea800a4fd0 tx=0x7fea800a6000 comp rx=0 tx=0).stop 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- 172.21.15.40:0/3589410044 shutdown_connections 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3589410044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16dd40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- 172.21.15.40:0/3589410044 >> 172.21.15.40:0/3589410044 conn(0x55b02ec36780 msgr2=0x55b02f150280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- 172.21.15.40:0/3589410044 shutdown_connections 2024-04-02T16:12:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- 172.21.15.40:0/3589410044 wait complete. 2024-04-02T16:12:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- start start 2024-04-02T16:12:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 Processor -- start 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f16dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef292a0 con 0x55b02f12acd0 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b02ef76fb0 con 0x55b02f155380 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55b02ef29440 con 0x55b02f155ca0 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea90d2c000 10 client.0 _ll_register_callbacks cb 0x7fea78016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T16:12:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59434/0 (socket says 172.21.15.40:59434) 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3644329653 learned_addr learned my addr 172.21.15.40:0/3644329653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 msgr2=0x55b02f16dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f16dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 msgr2=0x55b02f16d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 -- 172.21.15.40:0/3644329653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02ef251f0 con 0x55b02f12acd0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea877fe640 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8001c980 tx=0x7fea8001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57ea39c247d0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.784+0000 7fea667fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800d4060 con 0x55b02f12acd0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fea800c7740 con 0x55b02f12acd0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea800c72d0 con 0x55b02f12acd0 2024-04-02T16:12:24.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fea800da050 con 0x55b02f12acd0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 10 client.5190 fetch_fsmap learned FSMap version 14 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55b02ef192a0 con 0x55b02f12acd0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800c72d0 con 0x55b02f12acd0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 14) ==== 2284+0+0 (secure 0 0 0) 0x7fea800dd020 con 0x55b02f12acd0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 10 client.5190 fetch_fsmap finished waiting for FSMap version 14 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 10 client.5190 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 20 client.5190 populate_metadata read hostname 'smithi040' 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f1cec60 0x55b02f1d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 4 client.5190 mds_command: new command op to 4989 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 --> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] -- command(tid 0: {"prefix": "status"}) -- 0x55b02ef7fec0 con 0x55b02f1cec60 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea86ffd640 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f1cec60 0x55b02f1d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 10 client.5190 ms_handle_connect on v2:172.21.15.40:6832/2966573419 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea86ffd640 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f1cec60 0x55b02f1d1050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32e004ab575f033 server_cookie=9065a93fc3010c29 in_seq=0 out_seq=0 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 1 -- 172.21.15.40:0/3644329653 <== mds.0 v2:172.21.15.40:6832/2966573419 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b02ef7fec0 con 0x55b02f1cec60 2024-04-02T16:12:24.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 10 client.5190 handle_command_reply: tid=0 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 client.5190 shutdown 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 msgr2=0x55b02f16d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fea8001c980 tx=0x7fea8001ace0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 shutdown_connections 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:6832/2966573419,v1:172.21.15.40:6833/2966573419] conn(0x55b02f1cec60 0x55b02f1d1050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55b02f155ca0 0x55b02f16dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b02f155380 0x55b02f16d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 --2- 172.21.15.40:0/3644329653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b02f12acd0 0x55b02f16d060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea667fc640 0 client.5190 ms_handle_reset on v2:172.21.15.40:6832/2966573419 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 >> 172.21.15.40:0/3644329653 conn(0x55b02ec36780 msgr2=0x55b02f1248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 shutdown_connections 2024-04-02T16:12:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 1 -- 172.21.15.40:0/3644329653 wait complete. 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea90d2c000 20 client.5190 trim_cache size 0 max 16384 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "f7c926e5-333e-4244-9af2-414c6ebf5077", 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4989, 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 9.0957897509999999, 2024-04-02T16:12:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 14, 2024-04-02T16:12:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 29, 2024-04-02T16:12:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 29, 2024-04-02T16:12:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.991727941000001 2024-04-02T16:12:24.798 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 msgr2=0x7fea6806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 0x7fea6806a840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea800ad000 tx=0x7fea80008040 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 msgr2=0x7fea8809aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea70007ef0 tx=0x7fea7000e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 shutdown_connections 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea880b25b0 0x7fea88151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fea68068380 0x7fea6806a840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea880b1c10 0x7fea8809aed0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 --2- 172.21.15.40:0/3333268295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea880ae080 0x7fea88150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 >> 172.21.15.40:0/3333268295 conn(0x7fea880b8fd0 msgr2=0x7fea880b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 shutdown_connections 2024-04-02T16:12:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.788+0000 7fea8dbad640 1 -- 172.21.15.40:0/3333268295 wait complete. 2024-04-02T16:12:24.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-02T16:12:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 Processor -- start 2024-04-02T16:12:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 -- start start 2024-04-02T16:12:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 0x7fe8e014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e00b6eb0 con 0x7fe8e002c460 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e009b3e0 con 0x7fe8e00b55a0 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e7002640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e002c840 con 0x7fe8e00ac700 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8dffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e4d9e640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe8e002c460 0x7fe8e01456e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:53662/0 (socket says 172.21.15.40:53662) 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8dffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:55376/0 (socket says 172.21.15.40:55376) 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e4d9e640 1 -- 172.21.15.40:0/4256998942 learned_addr learned my addr 172.21.15.40:0/4256998942 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e559f640 1 --2- 172.21.15.40:0/4256998942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 0x7fe8e014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e559f640 1 -- 172.21.15.40:0/4256998942 >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 msgr2=0x7fe8e00ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e559f640 1 --2- 172.21.15.40:0/4256998942 >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e559f640 1 -- 172.21.15.40:0/4256998942 >> v1:172.21.15.204:6789/0 conn(0x7fe8e002c460 legacy=0x7fe8e01456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8e559f640 1 -- 172.21.15.40:0/4256998942 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e006e210 con 0x7fe8e00ac700 2024-04-02T16:12:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.920+0000 7fe8dffff640 1 --2- 172.21.15.40:0/4256998942 >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:12:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.924+0000 7fe8e559f640 1 --2- 172.21.15.40:0/4256998942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 0x7fe8e014ff80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009d90 tx=0x7fe8d801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca844e1e3e4f4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8df7fe640 1 -- 172.21.15.40:0/4256998942 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80b2070 con 0x7fe8e00ac700 2024-04-02T16:12:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8df7fe640 1 -- 172.21.15.40:0/4256998942 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d80a57a0 con 0x7fe8e00ac700 2024-04-02T16:12:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8df7fe640 1 -- 172.21.15.40:0/4256998942 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80a5b70 con 0x7fe8e00ac700 2024-04-02T16:12:24.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- 172.21.15.40:0/4256998942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 msgr2=0x7fe8e014ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- 172.21.15.40:0/4256998942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 0x7fe8e014ff80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009d90 tx=0x7fe8d801c8e0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- 172.21.15.40:0/4256998942 shutdown_connections 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- 172.21.15.40:0/4256998942 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00ac700 0x7fe8e014ff80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- 172.21.15.40:0/4256998942 >> v2:172.21.15.204:3300/0 conn(0x7fe8e00b55a0 0x7fe8e00ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- 172.21.15.40:0/4256998942 >> 172.21.15.40:0/4256998942 conn(0x7fe8e00b5f60 msgr2=0x7fe8e00b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- 172.21.15.40:0/4256998942 shutdown_connections 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- 172.21.15.40:0/4256998942 wait complete. 2024-04-02T16:12:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 Processor -- start 2024-04-02T16:12:24.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- start start 2024-04-02T16:12:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 0x7fe8e0091bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00b55a0 0x7fe8e01de050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e009c050 con 0x7fe8e002c460 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e009bed0 con 0x7fe8e00ac700 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e7002640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e0152ce0 con 0x7fe8e00b55a0 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59448/0 (socket says 172.21.15.40:59448) 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 -- 172.21.15.40:0/290828517 learned_addr learned my addr 172.21.15.40:0/290828517 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8dffff640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 0x7fe8e0091bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e559f640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00b55a0 0x7fe8e01de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 -- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00b55a0 msgr2=0x7fe8e01de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00b55a0 0x7fe8e01de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 -- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 msgr2=0x7fe8e0091bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 0x7fe8e0091bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:24.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 -- 172.21.15.40:0/290828517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e00b9100 con 0x7fe8e002c460 2024-04-02T16:12:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8dffff640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 0x7fe8e0091bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8e4d9e640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c420 tx=0x7fe8d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4017070 con 0x7fe8e002c460 2024-04-02T16:12:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4005cf0 con 0x7fe8e002c460 2024-04-02T16:12:24.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.928+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40072f0 con 0x7fe8e002c460 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e01e1860 con 0x7fe8e002c460 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01e1e20 con 0x7fe8e002c460 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e0068d00 con 0x7fe8e002c460 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fe8d401a070 con 0x7fe8e002c460 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8dd7fa640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 0x7fe8c406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:24.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fe8d4085d90 con 0x7fe8e002c460 2024-04-02T16:12:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8dffff640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 0x7fe8c406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:24.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.932+0000 7fe8dffff640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 0x7fe8c406a840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0091b50 tx=0x7fe8cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:24.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:24.936+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fe8e00b6eb0 con 0x7fe8e002c460 2024-04-02T16:12:25.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.132+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8e01e1ff0 con 0x7fe8e002c460 2024-04-02T16:12:25.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.132+0000 7fe8dd7fa640 1 -- 172.21.15.40:0/290828517 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fe8d4056340 con 0x7fe8e002c460 2024-04-02T16:12:25.135 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-02T16:12:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 msgr2=0x7fe8c406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:25.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 0x7fe8c406a840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0091b50 tx=0x7fe8cc008040 comp rx=0 tx=0).stop 2024-04-02T16:12:25.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 msgr2=0x7fe8e00985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:25.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400c420 tx=0x7fe8d400c8e0 comp rx=0 tx=0).stop 2024-04-02T16:12:25.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 shutdown_connections 2024-04-02T16:12:25.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8e00b55a0 0x7fe8e01de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:25.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fe8c4068380 0x7fe8c406a840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:25.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8e00ac700 0x7fe8e0091bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:25.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 --2- 172.21.15.40:0/290828517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8e002c460 0x7fe8e00985e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:25.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 >> 172.21.15.40:0/290828517 conn(0x7fe8e00b5f60 msgr2=0x7fe8e006cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:25.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 shutdown_connections 2024-04-02T16:12:25.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25.136+0000 7fe8e7002640 1 -- 172.21.15.40:0/290828517 wait complete. 2024-04-02T16:12:25.152 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T16:12:25.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T16:12:25.153 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:12:25.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.153 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.160 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42453sec preferred_lft 42453sec 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.161 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:e4:3d:72:cc brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:25.162 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-02T16:12:25.163 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-02T16:12:25.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.163 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T16:12:25.212 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-02T16:12:25.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.214 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-02T16:12:25.261 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-02T16:12:25.262 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T16:12:25.262 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-02T16:12:25.262 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-02T16:12:25.262 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-02T16:12:25.262 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T16:12:25.262 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T16:12:25.262 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-02T16:12:25.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-02T16:12:25.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-02T16:12:25.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-02T16:12:25.263 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-02T16:12:25.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T16:12:25.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:12:25.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.381 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-02T16:12:25.428 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T16:12:25.429 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:12:25.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-02T16:12:25.484 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-02T16:12:25.484 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:12:25.484 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:25.532 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T16:12:25.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T16:12:25.533 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:25.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:12:25.637 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:12:25.685 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:12:25.685 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:12:25.685 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:12:25.685 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:12:25.685 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:12:25.686 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:25.687 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:25.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.688 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:12:25.736 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T16:12:25.736 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-02T16:12:25.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:12:25.796 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:12:25.844 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:12:25.845 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:12:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:12:25.847 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:12:25.847 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:25.847 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:25.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:25.848 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:12:25.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2024-04-02T16:12:25.864+0000 7fc4c3cb9580 -1 init, newargv = 0x561ffeb688f0 newargc=15 2024-04-02T16:12:25.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[36834]: starting ceph client 2024-04-02T16:12:25.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[36834]: starting fuse 2024-04-02T16:12:26.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-02T16:12:26.988 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T16:12:27.037 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-02T16:12:27.037 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-02T16:12:27.037 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-02T16:12:27.037 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-02T16:12:27.037 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-02T16:12:27.038 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:27.039 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:27.040 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:27.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:27.040 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-02T16:12:27.088 INFO:teuthology.orchestra.run.smithi040.stdout:45 2024-04-02T16:12:27.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-04-02T16:12:27.089 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.0")) 2024-04-02T16:12:27.090 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-02T16:12:27.216 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.36834.asok 2024-04-02T16:12:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:12:27.227 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36834.asok 2024-04-02T16:12:27.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:27.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36834.asok status 2024-04-02T16:12:27.388 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "36834", 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5199, 2024-04-02T16:12:27.389 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5199 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3355100940 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-02T16:12:27.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3355100940 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.5199 192.168.0.1:0/3355100940", 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/3355100940", 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 14, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 29, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-04-02T16:12:27.391 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:27.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:27.402 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-04-02T16:12:27.403 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:27.409 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-04-02T16:12:27.409 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:27.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:27.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:27.465 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-02T16:12:27.512 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.qbw79Iz5Id 2024-04-02T16:12:27.512 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:12:27.513 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/tmp/tmp.qbw79Iz5Id 2024-04-02T16:12:27.560 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-04-02T16:12:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:12:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:12:27.749 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfsza35oz2z') 2024-04-02T16:12:27.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T16:12:27.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T16:12:27.819 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfsza35oz2z" 2024-04-02T16:12:27.819 DEBUG:teuthology.orchestra.run.smithi040:> cephfs-shell -b /tmp/test-cephfsza35oz2z 2024-04-02T16:12:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:Unrecognized command 2024-04-02T16:12:28.173 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-04-02T16:12:28.174 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-02T16:12:28.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:12:28.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-02T16:12:28.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-02T16:12:28.294 INFO:tasks.cephfs.mount:Terminating background process 2024-04-02T16:12:28.294 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:12:28.294 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:12:28.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.472+0000 7fd42789d640 1 Processor -- start 2024-04-02T16:12:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.472+0000 7fd42789d640 1 -- start start 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd42789d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd420073230 0x7fd4200b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd42789d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd42789d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200b19e0 con 0x7fd4200b23d0 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd42789d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd4201497c0 con 0x7fd420073230 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd42789d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4200b05c0 con 0x7fd420148e50 2024-04-02T16:12:28.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd425e3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd425e3a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:38694/0 (socket says 172.21.15.40:38694) 2024-04-02T16:12:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd425e3a640 1 -- 172.21.15.40:0/3442439307 learned_addr learned my addr 172.21.15.40:0/3442439307 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 --2- 172.21.15.40:0/3442439307 >> v2:172.21.15.204:3300/0 conn(0x7fd420073230 0x7fd4200b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 -- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 msgr2=0x7fd420149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 -- 172.21.15.40:0/3442439307 >> v1:172.21.15.204:6789/0 conn(0x7fd4200b23d0 legacy=0x7fd42009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 -- 172.21.15.40:0/3442439307 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200b08a0 con 0x7fd420073230 2024-04-02T16:12:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd425e3a640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:12:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd424e38640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd420073230 0x7fd4200b27b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd4140097c0 tx=0x7fd41401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24db53cd6348c95c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd41ffff640 1 -- 172.21.15.40:0/3442439307 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4140ab430 con 0x7fd420073230 2024-04-02T16:12:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd41ffff640 1 -- 172.21.15.40:0/3442439307 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4140b0070 con 0x7fd420073230 2024-04-02T16:12:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.476+0000 7fd41ffff640 1 -- 172.21.15.40:0/3442439307 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4140ab7f0 con 0x7fd420073230 2024-04-02T16:12:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd420073230 msgr2=0x7fd4200b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd420073230 0x7fd4200b27b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd4140097c0 tx=0x7fd41401c410 comp rx=0 tx=0).stop 2024-04-02T16:12:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/3442439307 shutdown_connections 2024-04-02T16:12:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420148e50 0x7fd420149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- 172.21.15.40:0/3442439307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd420073230 0x7fd4200b27b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/3442439307 >> 172.21.15.40:0/3442439307 conn(0x7fd4200b8fd0 msgr2=0x7fd4200b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/3442439307 shutdown_connections 2024-04-02T16:12:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/3442439307 wait complete. 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 Processor -- start 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- start start 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 0x7fd42013e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4200b23d0 0x7fd42013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd420152810 con 0x7fd420148e50 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd420152690 con 0x7fd4200b23d0 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd420152990 con 0x7fd420073230 2024-04-02T16:12:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40660/0 (socket says 172.21.15.40:40660) 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 -- 172.21.15.40:0/410966659 learned_addr learned my addr 172.21.15.40:0/410966659 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425639640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 0x7fd42013e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd424e38640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4200b23d0 0x7fd42013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 -- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 msgr2=0x7fd42013e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 0x7fd42013e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 -- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4200b23d0 msgr2=0x7fd42013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4200b23d0 0x7fd42013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 -- 172.21.15.40:0/410966659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201380b0 con 0x7fd420148e50 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425639640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 0x7fd42013e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd425e3a640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd41800e960 tx=0x7fd41800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418017070 con 0x7fd420148e50 2024-04-02T16:12:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41800ca00 con 0x7fd420148e50 2024-04-02T16:12:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd420138390 con 0x7fd420148e50 2024-04-02T16:12:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41800cdd0 con 0x7fd420148e50 2024-04-02T16:12:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.480+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42013f6f0 con 0x7fd420148e50 2024-04-02T16:12:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42013d1f0 con 0x7fd420148e50 2024-04-02T16:12:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd418013470 con 0x7fd420148e50 2024-04-02T16:12:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd41dffb640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 0x7fd40c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd418086ab0 con 0x7fd420148e50 2024-04-02T16:12:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd425639640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 0x7fd40c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.484+0000 7fd425639640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 0x7fd40c06a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd410001e10 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.488+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd418057100 con 0x7fd420148e50 2024-04-02T16:12:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.652+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4200685f0 con 0x7fd420148e50 2024-04-02T16:12:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.652+0000 7fd41dffb640 1 -- 172.21.15.40:0/410966659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16621 (secure 0 0 0) 0x7fd418056a00 con 0x7fd420148e50 2024-04-02T16:12:28.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:28.655 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:16.692754+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 msgr2=0x7fd40c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 0x7fd40c06a840 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd410001e10 tx=0x7fd410009040 comp rx=0 tx=0).stop 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 msgr2=0x7fd4201388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd41800e960 tx=0x7fd41800ee20 comp rx=0 tx=0).stop 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 shutdown_connections 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd40c068380 0x7fd40c06a840 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd420148e50 0x7fd4201388b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4200b23d0 0x7fd42013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 --2- 172.21.15.40:0/410966659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd420073230 0x7fd42013e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 >> 172.21.15.40:0/410966659 conn(0x7fd4200b8fd0 msgr2=0x7fd4200b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 shutdown_connections 2024-04-02T16:12:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.656+0000 7fd42789d640 1 -- 172.21.15.40:0/410966659 wait complete. 2024-04-02T16:12:28.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:12:13.528574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:12:14.023094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:12:28.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:12:28.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 Processor -- start 2024-04-02T16:12:28.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 -- start start 2024-04-02T16:12:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd50406d610 0x7fd504144460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5040af4f0 0x7fd50414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd504145390 con 0x7fd50406d9f0 2024-04-02T16:12:28.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040af960 con 0x7fd5040af4f0 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd50a587640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040b62e0 con 0x7fd50406d610 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd5037fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd50406d9f0 0x7fd504144c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54134/0 (socket says 172.21.15.40:54134) 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd5037fe640 1 -- 172.21.15.40:0/3839861556 learned_addr learned my addr 172.21.15.40:0/3839861556 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd508b24640 1 --2- 172.21.15.40:0/3839861556 >> v2:172.21.15.40:3300/0 conn(0x7fd5040af4f0 0x7fd50414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd503fff640 1 --2- 172.21.15.40:0/3839861556 >> v2:172.21.15.204:3300/0 conn(0x7fd50406d610 0x7fd504144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd502ffd640 1 -- 172.21.15.40:0/3839861556 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2486950501 0 0) 0x7fd504145390 con 0x7fd50406d9f0 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd502ffd640 1 -- 172.21.15.40:0/3839861556 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4ec003530 con 0x7fd50406d9f0 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd508b24640 1 -- 172.21.15.40:0/3839861556 >> v2:172.21.15.204:3300/0 conn(0x7fd50406d610 msgr2=0x7fd504144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd508b24640 1 --2- 172.21.15.40:0/3839861556 >> v2:172.21.15.204:3300/0 conn(0x7fd50406d610 0x7fd504144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd508b24640 1 -- 172.21.15.40:0/3839861556 >> v1:172.21.15.40:6789/0 conn(0x7fd50406d9f0 legacy=0x7fd504144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.796+0000 7fd508b24640 1 -- 172.21.15.40:0/3839861556 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040b65c0 con 0x7fd5040af4f0 2024-04-02T16:12:28.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd508b24640 1 --2- 172.21.15.40:0/3839861556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5040af4f0 0x7fd50414ed20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd5040b1170 tx=0x7fd4fc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a274340e6ceb4b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd502ffd640 1 -- 172.21.15.40:0/3839861556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0b0070 con 0x7fd5040af4f0 2024-04-02T16:12:28.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd502ffd640 1 -- 172.21.15.40:0/3839861556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4fc0ab600 con 0x7fd5040af4f0 2024-04-02T16:12:28.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd502ffd640 1 -- 172.21.15.40:0/3839861556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0ab9d0 con 0x7fd5040af4f0 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 -- 172.21.15.40:0/3839861556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5040af4f0 msgr2=0x7fd50414ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 --2- 172.21.15.40:0/3839861556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5040af4f0 0x7fd50414ed20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd5040b1170 tx=0x7fd4fc01c3f0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 -- 172.21.15.40:0/3839861556 shutdown_connections 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 --2- 172.21.15.40:0/3839861556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5040af4f0 0x7fd50414ed20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 --2- 172.21.15.40:0/3839861556 >> v2:172.21.15.204:3300/0 conn(0x7fd50406d610 0x7fd504144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 -- 172.21.15.40:0/3839861556 >> 172.21.15.40:0/3839861556 conn(0x7fd50402cdd0 msgr2=0x7fd5040b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 -- 172.21.15.40:0/3839861556 shutdown_connections 2024-04-02T16:12:28.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.800+0000 7fd50a587640 1 -- 172.21.15.40:0/3839861556 wait complete. 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 Processor -- start 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- start start 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd50406d9f0 0x7fd504071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5040af4f0 0x7fd50409bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd50406eb50 con 0x7fd50406d610 2024-04-02T16:12:28.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd50409f2d0 con 0x7fd5040af4f0 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd50406ecd0 con 0x7fd50406d9f0 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40684/0 (socket says 172.21.15.40:40684) 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 -- 172.21.15.40:0/1919942898 learned_addr learned my addr 172.21.15.40:0/1919942898 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd5037fe640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd50406d9f0 0x7fd504071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd508b24640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5040af4f0 0x7fd50409bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 -- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd50406d9f0 msgr2=0x7fd504071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd50406d9f0 0x7fd504071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 -- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5040af4f0 msgr2=0x7fd50409bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5040af4f0 0x7fd50409bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 -- 172.21.15.40:0/1919942898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50409ac10 con 0x7fd50406d610 2024-04-02T16:12:28.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd503fff640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007e70 tx=0x7fd4f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4014620 con 0x7fd50406d610 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f4019070 con 0x7fd50406d610 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50409c5d0 con 0x7fd50406d610 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4014a00 con 0x7fd50406d610 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50409b340 con 0x7fd50406d610 2024-04-02T16:12:28.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5040685f0 con 0x7fd50406d610 2024-04-02T16:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd4f4014ba0 con 0x7fd50406d610 2024-04-02T16:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 0x7fd4f006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.804+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd4f40871a0 con 0x7fd50406d610 2024-04-02T16:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.808+0000 7fd5037fe640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 0x7fd4f006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:28.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.808+0000 7fd5037fe640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 0x7fd4f006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5041e2e90 tx=0x7fd4f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:28.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.812+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd4f4011260 con 0x7fd50406d610 2024-04-02T16:12:28.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.972+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd50406cf40 con 0x7fd50406d610 2024-04-02T16:12:28.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.972+0000 7fd500ff9640 1 -- 172.21.15.40:0/1919942898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fd4f40577f0 con 0x7fd50406d610 2024-04-02T16:12:28.974 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:12:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 msgr2=0x7fd4f006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 0x7fd4f006a840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5041e2e90 tx=0x7fd4f8009040 comp rx=0 tx=0).stop 2024-04-02T16:12:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 msgr2=0x7fd5040b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4007e70 tx=0x7fd4f400e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 shutdown_connections 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd50406d9f0 0x7fd504071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd4f0068380 0x7fd4f006a840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5040af4f0 0x7fd50409bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 --2- 172.21.15.40:0/1919942898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd50406d610 0x7fd5040b3690 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 >> 172.21.15.40:0/1919942898 conn(0x7fd50402cdd0 msgr2=0x7fd5040b9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 shutdown_connections 2024-04-02T16:12:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:28.976+0000 7fd50a587640 1 -- 172.21.15.40:0/1919942898 wait complete. 2024-04-02T16:12:28.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:12:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 Processor -- start 2024-04-02T16:12:29.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 -- start start 2024-04-02T16:12:29.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8568148550 0x7f85680af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8568141460 0x7f8568141840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8568071550 con 0x7f8568072910 2024-04-02T16:12:29.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8568147a70 con 0x7f8568148550 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856e4c0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8568147d50 con 0x7f8568141460 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8568148550 0x7f85680af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8568148550 0x7f85680af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40690/0 (socket says 172.21.15.40:40690) 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 -- 172.21.15.40:0/1801386955 learned_addr learned my addr 172.21.15.40:0/1801386955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f856ca5d640 1 --2- 172.21.15.40:0/1801386955 >> v2:172.21.15.204:3300/0 conn(0x7f8568141460 0x7f8568141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 -- 172.21.15.40:0/1801386955 >> v2:172.21.15.204:3300/0 conn(0x7f8568141460 msgr2=0x7f8568141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 --2- 172.21.15.40:0/1801386955 >> v2:172.21.15.204:3300/0 conn(0x7f8568141460 0x7f8568141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 -- 172.21.15.40:0/1801386955 >> v1:172.21.15.204:6789/0 conn(0x7f8568072910 legacy=0x7f8568070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.120+0000 7f85677fe640 1 -- 172.21.15.40:0/1801386955 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8568148030 con 0x7f8568148550 2024-04-02T16:12:29.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f85677fe640 1 --2- 172.21.15.40:0/1801386955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568148550 0x7f85680af1e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8568070ce0 tx=0x7f855c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be37d8578fefe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f8566ffd640 1 -- 172.21.15.40:0/1801386955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0a5770 con 0x7f8568148550 2024-04-02T16:12:29.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f8566ffd640 1 -- 172.21.15.40:0/1801386955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c0a5910 con 0x7f8568148550 2024-04-02T16:12:29.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f8566ffd640 1 -- 172.21.15.40:0/1801386955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0a5ce0 con 0x7f8568148550 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- 172.21.15.40:0/1801386955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568148550 msgr2=0x7f85680af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 --2- 172.21.15.40:0/1801386955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568148550 0x7f85680af1e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8568070ce0 tx=0x7f855c01c890 comp rx=0 tx=0).stop 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- 172.21.15.40:0/1801386955 shutdown_connections 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 --2- 172.21.15.40:0/1801386955 >> v2:172.21.15.204:3300/0 conn(0x7f8568141460 0x7f8568141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 --2- 172.21.15.40:0/1801386955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568148550 0x7f85680af1e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- 172.21.15.40:0/1801386955 >> 172.21.15.40:0/1801386955 conn(0x7f85680b7f50 msgr2=0x7f85680b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- 172.21.15.40:0/1801386955 shutdown_connections 2024-04-02T16:12:29.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- 172.21.15.40:0/1801386955 wait complete. 2024-04-02T16:12:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 Processor -- start 2024-04-02T16:12:29.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.124+0000 7f856e4c0640 1 -- start start 2024-04-02T16:12:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 0x7f8568136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 0x7f85681e1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8568151670 con 0x7f8568141460 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85681514f0 con 0x7f8568072910 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f85681517f0 con 0x7f8568148550 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40696/0 (socket says 172.21.15.40:40696) 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 -- 172.21.15.40:0/41863432 learned_addr learned my addr 172.21.15.40:0/41863432 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8567fff640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 0x7f8568136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856ca5d640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 0x7f85681e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 -- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 msgr2=0x7f85681e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 0x7f85681e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 -- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 msgr2=0x7f8568136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 0x7f8568136f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 -- 172.21.15.40:0/41863432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856813a530 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8567fff640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 0x7f8568136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856ca5d640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 0x7f85681e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f85677fe640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f855c0af040 tx=0x7f855c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0bc020 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c0b8670 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85681379f0 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c0b8a60 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8568147340 con 0x7f8568141460 2024-04-02T16:12:29.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8568068d00 con 0x7f8568141460 2024-04-02T16:12:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f855c0bf020 con 0x7f8568141460 2024-04-02T16:12:29.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 0x7f855406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.128+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f855c121cb0 con 0x7f8568141460 2024-04-02T16:12:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.132+0000 7f8567fff640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 0x7f855406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.132+0000 7f8567fff640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 0x7f855406a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8560003e40 tx=0x7f8560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.132+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f855c0c6030 con 0x7f8568141460 2024-04-02T16:12:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.296+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f856813a7a0 con 0x7f8568141460 2024-04-02T16:12:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.296+0000 7f8564ff9640 1 -- 172.21.15.40:0/41863432 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f855c0165a0 con 0x7f8568141460 2024-04-02T16:12:29.300 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:12:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 msgr2=0x7f855406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 0x7f855406a840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8560003e40 tx=0x7f8560009040 comp rx=0 tx=0).stop 2024-04-02T16:12:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 msgr2=0x7f85681374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f855c0af040 tx=0x7f855c019630 comp rx=0 tx=0).stop 2024-04-02T16:12:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 shutdown_connections 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8568148550 0x7f85681e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8554068380 0x7f855406a840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8568141460 0x7f85681374b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 --2- 172.21.15.40:0/41863432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8568072910 0x7f8568136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 >> 172.21.15.40:0/41863432 conn(0x7f85680b7f50 msgr2=0x7f85680b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 shutdown_connections 2024-04-02T16:12:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.300+0000 7f856e4c0640 1 -- 172.21.15.40:0/41863432 wait complete. 2024-04-02T16:12:29.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:12:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 Processor -- start 2024-04-02T16:12:29.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 -- start start 2024-04-02T16:12:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7600b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb760144980 0x7fb76006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb760143d10 con 0x7fb76006d130 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600ae7c0 con 0x7fb760144980 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb766b27640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600af060 con 0x7fb760144520 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb7648c3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb76006d130 0x7fb76009dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54146/0 (socket says 172.21.15.40:54146) 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb7648c3640 1 -- 172.21.15.40:0/1642029221 learned_addr learned my addr 172.21.15.40:0/1642029221 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831139320 0 0) 0x7fb760143d10 con 0x7fb76006d130 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb740003530 con 0x7fb76006d130 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb75ffff640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7600b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.440+0000 7fb7650c4640 1 --2- 172.21.15.40:0/1642029221 >> v2:172.21.15.204:3300/0 conn(0x7fb760144980 0x7fb76006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2741471514 0 0) 0x7fb740003530 con 0x7fb76006d130 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 msgr2=0x7fb7600b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7600b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 >> v2:172.21.15.204:3300/0 conn(0x7fb760144980 msgr2=0x7fb76006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 --2- 172.21.15.40:0/1642029221 >> v2:172.21.15.204:3300/0 conn(0x7fb760144980 0x7fb76006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76009e3c0 con 0x7fb76006d130 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fb750001680 con 0x7fb76006d130 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 >> v1:172.21.15.40:6789/0 conn(0x7fb76006d130 legacy=0x7fb76009dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7400039c0 0x7fb740003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb740004810 0x7fb740004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600ae7c0 con 0x7fb7400039c0 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600af060 con 0x7fb740004810 2024-04-02T16:12:29.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb74000bab0 con 0x7fb760144520 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7400039c0 0x7fb740003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 -- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 msgr2=0x7fb7400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7650c4640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75ffff640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb740004810 0x7fb740004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb760144520 0x7fb7400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 -- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb740004810 msgr2=0x7fb740004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb740004810 0x7fb740004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 -- 172.21.15.40:0/1642029221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb740003530 con 0x7fb7400039c0 2024-04-02T16:12:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7648c3640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7400039c0 0x7fb740003de0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb750001ec0 tx=0x7fb75001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c6385072208e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750007cb0 con 0x7fb7400039c0 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7500a9020 con 0x7fb7400039c0 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb75f7fe640 1 -- 172.21.15.40:0/1642029221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750005b20 con 0x7fb7400039c0 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 -- 172.21.15.40:0/1642029221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7400039c0 msgr2=0x7fb740003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 --2- 172.21.15.40:0/1642029221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7400039c0 0x7fb740003de0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb750001ec0 tx=0x7fb75001cb80 comp rx=0 tx=0).stop 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb7650c4640 1 -- 172.21.15.40:0/1642029221 reap_dead start 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 -- 172.21.15.40:0/1642029221 shutdown_connections 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 -- 172.21.15.40:0/1642029221 >> 172.21.15.40:0/1642029221 conn(0x7fb7600b8fd0 msgr2=0x7fb7600b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 -- 172.21.15.40:0/1642029221 shutdown_connections 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.444+0000 7fb766b27640 1 -- 172.21.15.40:0/1642029221 wait complete. 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 Processor -- start 2024-04-02T16:12:29.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- start start 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7400039c0 0x7fb76014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb76014e750 0x7fb76014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb76006cb80 con 0x7fb7601ddcc0 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600ae300 con 0x7fb7400039c0 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb76006cd00 con 0x7fb76014e750 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40714/0 (socket says 172.21.15.40:40714) 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 -- 172.21.15.40:0/2262873286 learned_addr learned my addr 172.21.15.40:0/2262873286 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75ffff640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb76014e750 0x7fb76014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7648c3640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7400039c0 0x7fb76014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 -- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb76014e750 msgr2=0x7fb76014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb76014e750 0x7fb76014ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 -- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7400039c0 msgr2=0x7fb76014c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7400039c0 0x7fb76014c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 -- 172.21.15.40:0/2262873286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760143eb0 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb7650c4640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb76006e9a0 tx=0x7fb758010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800f040 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75800e8a0 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb760142f50 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800ec90 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb760142c70 con 0x7fb7601ddcc0 2024-04-02T16:12:29.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7600685f0 con 0x7fb7601ddcc0 2024-04-02T16:12:29.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb7580253f0 con 0x7fb7601ddcc0 2024-04-02T16:12:29.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.448+0000 7fb75d7fa640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 0x7fb74406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.452+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fb758086c10 con 0x7fb7601ddcc0 2024-04-02T16:12:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.452+0000 7fb7648c3640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 0x7fb74406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.452+0000 7fb7648c3640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 0x7fb74406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7500aa000 tx=0x7fb750001980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.456+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb758018030 con 0x7fb7601ddcc0 2024-04-02T16:12:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.616+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb760142020 con 0x7fb7601ddcc0 2024-04-02T16:12:29.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.616+0000 7fb75d7fa640 1 -- 172.21.15.40:0/2262873286 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fb758057260 con 0x7fb7601ddcc0 2024-04-02T16:12:29.618 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 msgr2=0x7fb74406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 0x7fb74406a840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb7500aa000 tx=0x7fb750001980 comp rx=0 tx=0).stop 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 msgr2=0x7fb7601de140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb76006e9a0 tx=0x7fb758010b70 comp rx=0 tx=0).stop 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 shutdown_connections 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb76014e750 0x7fb76014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb744068380 0x7fb74406a840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7601ddcc0 0x7fb7601de140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 --2- 172.21.15.40:0/2262873286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7400039c0 0x7fb76014c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 >> 172.21.15.40:0/2262873286 conn(0x7fb7600b8fd0 msgr2=0x7fb7600b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 shutdown_connections 2024-04-02T16:12:29.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.620+0000 7fb766b27640 1 -- 172.21.15.40:0/2262873286 wait complete. 2024-04-02T16:12:29.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T16:12:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.760+0000 7f205031c640 1 Processor -- start 2024-04-02T16:12:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.760+0000 7f205031c640 1 -- start start 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.760+0000 7f205031c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f2048150e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.760+0000 7f205031c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20480b5610 con 0x7f20480b4060 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.760+0000 7f205031c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20480b6450 con 0x7f20480b6070 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f205031c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2048073330 con 0x7f20480b47c0 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d8b7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f20480b4060 0x7f2048145950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54160/0 (socket says 172.21.15.40:54160) 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d8b7640 1 -- 172.21.15.40:0/51739278 learned_addr learned my addr 172.21.15.40:0/51739278 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f2048150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068463593 0 0) 0x7f20480b5610 con 0x7f20480b4060 2024-04-02T16:12:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2034003530 con 0x7f20480b4060 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1078652491 0 0) 0x7f2034003530 con 0x7f20480b4060 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 msgr2=0x7f2048150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f2048150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 >> v1:172.21.15.204:6789/0 conn(0x7f20480b6070 legacy=0x7f20480b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048072d70 con 0x7f20480b4060 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f20380016a0 con 0x7f20480b4060 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 >> v1:172.21.15.40:6789/0 conn(0x7f20480b4060 legacy=0x7f2048145950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f20340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034003f30 0x7f2034004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2034004d50 0x7f20340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2048073330 con 0x7f2034004d50 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f203400bb90 con 0x7f2034003f30 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f203400bd90 con 0x7f20480b47c0 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2034004d50 0x7f20340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d8b7640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034003f30 0x7f2034004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e0b8640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f20340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 -- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 msgr2=0x7f20340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20480b47c0 0x7f20340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 -- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034003f30 msgr2=0x7f2034004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:29.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034003f30 0x7f2034004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 -- 172.21.15.40:0/51739278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2034003530 con 0x7f2034004d50 2024-04-02T16:12:29.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204e8b9640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2034004d50 0x7f20340051d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2040009ab0 tx=0x7f204001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afb4a3be3a18d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20400b2070 con 0x7f2034004d50 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20400a5640 con 0x7f2034004d50 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.764+0000 7f204d0b6640 1 -- 172.21.15.40:0/51739278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20400096b0 con 0x7f2034004d50 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/51739278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2034004d50 msgr2=0x7f20340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 --2- 172.21.15.40:0/51739278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2034004d50 0x7f20340051d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2040009ab0 tx=0x7f204001ca50 comp rx=0 tx=0).stop 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 -- 172.21.15.40:0/51739278 reap_dead start 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/51739278 shutdown_connections 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/51739278 >> 172.21.15.40:0/51739278 conn(0x7f204806e600 msgr2=0x7f204806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/51739278 shutdown_connections 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/51739278 wait complete. 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 Processor -- start 2024-04-02T16:12:29.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- start start 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 0x7f204813cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 0x7f2048071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f20480b5160 con 0x7f20480b7bc0 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2048072a40 con 0x7f2034004d50 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f20480b52e0 con 0x7f20481dfd50 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40736/0 (socket says 172.21.15.40:40736) 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 -- 172.21.15.40:0/622118427 learned_addr learned my addr 172.21.15.40:0/622118427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204d8b7640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 0x7f2048071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 -- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 msgr2=0x7f2048071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 0x7f2048071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 -- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 msgr2=0x7f204813cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e0b8640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 0x7f204813cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 0x7f204813cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 -- 172.21.15.40:0/622118427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048072020 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204d8b7640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 0x7f2048071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e0b8640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 0x7f204813cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f204e8b9640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2040009ab0 tx=0x7f2034005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20400b2040 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040000e50 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2048070a40 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040004280 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20480adc70 con 0x7f20480b7bc0 2024-04-02T16:12:29.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.768+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2048070830 con 0x7f20480b7bc0 2024-04-02T16:12:29.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.772+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f20400be060 con 0x7f20480b7bc0 2024-04-02T16:12:29.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.772+0000 7f203effd640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 0x7f202c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:29.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.772+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f204011fdb0 con 0x7f20480b7bc0 2024-04-02T16:12:29.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.772+0000 7f204e0b8640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 0x7f202c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:29.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.772+0000 7f204e0b8640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 0x7f202c06a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2044007fd0 tx=0x7f2044007f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:29.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.776+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f2040016760 con 0x7f20480b7bc0 2024-04-02T16:12:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.944+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2048068d00 con 0x7f20480b7bc0 2024-04-02T16:12:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.944+0000 7f203effd640 1 -- 172.21.15.40:0/622118427 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f20400f04e0 con 0x7f20480b7bc0 2024-04-02T16:12:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-04-02T16:12:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 msgr2=0x7f202c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 0x7f202c06a840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2044007fd0 tx=0x7f2044007f60 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 msgr2=0x7f20480b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2040009ab0 tx=0x7f2034005840 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 shutdown_connections 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20481dfd50 0x7f2048071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f202c068380 0x7f202c06a840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20480b7bc0 0x7f20480b8040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 --2- 172.21.15.40:0/622118427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2034004d50 0x7f204813cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 >> 172.21.15.40:0/622118427 conn(0x7f204806e600 msgr2=0x7f2048145850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 shutdown_connections 2024-04-02T16:12:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:29.948+0000 7f205031c640 1 -- 172.21.15.40:0/622118427 wait complete. 2024-04-02T16:12:29.963 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-04-02T16:12:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:29.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-02T16:12:29.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[36834]: fuse finished with error 0 and tester_r 0 2024-04-02T16:12:29.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:12:29.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:29.976 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T16:12:30.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:30.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 Processor -- start 2024-04-02T16:12:30.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 -- start start 2024-04-02T16:12:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5bc0b6170 0x7fb5bc148e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 0x7fb5bc14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5bc0b52e0 con 0x7fb5bc0b4f00 2024-04-02T16:12:30.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc0b6f00 con 0x7fb5bc0b6170 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c3073640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc0b71a0 con 0x7fb5bc1493c0 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5bc0b6170 0x7fb5bc148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5bc0b6170 0x7fb5bc148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40740/0 (socket says 172.21.15.40:40740) 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 -- 172.21.15.40:0/847907790 learned_addr learned my addr 172.21.15.40:0/847907790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c1610640 1 --2- 172.21.15.40:0/847907790 >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 0x7fb5bc14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 -- 172.21.15.40:0/847907790 >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 msgr2=0x7fb5bc14fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:30.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 --2- 172.21.15.40:0/847907790 >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 0x7fb5bc14fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 -- 172.21.15.40:0/847907790 >> v1:172.21.15.204:6789/0 conn(0x7fb5bc0b4f00 legacy=0x7fb5bc0bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:30.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5bbfff640 1 -- 172.21.15.40:0/847907790 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc1497d0 con 0x7fb5bc0b6170 2024-04-02T16:12:30.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.224+0000 7fb5c1610640 1 --2- 172.21.15.40:0/847907790 >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 0x7fb5bc14fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:30.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 --2- 172.21.15.40:0/847907790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc148e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b6730 tx=0x7fb5b001c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9f3664f2dc1b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bb7fe640 1 -- 172.21.15.40:0/847907790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b0070 con 0x7fb5bc0b6170 2024-04-02T16:12:30.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bb7fe640 1 -- 172.21.15.40:0/847907790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b00ab600 con 0x7fb5bc0b6170 2024-04-02T16:12:30.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bb7fe640 1 -- 172.21.15.40:0/847907790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00ab9d0 con 0x7fb5bc0b6170 2024-04-02T16:12:30.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- 172.21.15.40:0/847907790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 msgr2=0x7fb5bc148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- 172.21.15.40:0/847907790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc148e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0b6730 tx=0x7fb5b001c1e0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- 172.21.15.40:0/847907790 shutdown_connections 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- 172.21.15.40:0/847907790 >> v2:172.21.15.204:3300/0 conn(0x7fb5bc1493c0 0x7fb5bc14fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- 172.21.15.40:0/847907790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc148e80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- 172.21.15.40:0/847907790 >> 172.21.15.40:0/847907790 conn(0x7fb5bc02c5d0 msgr2=0x7fb5bc02c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- 172.21.15.40:0/847907790 shutdown_connections 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- 172.21.15.40:0/847907790 wait complete. 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 Processor -- start 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- start start 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 0x7fb5bc13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 0x7fb5bc13c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc072b90 con 0x7fb5bc0b6170 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b59e0 con 0x7fb5bc0b4f00 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c3073640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc152ba0 con 0x7fb5bc1493c0 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40742/0 (socket says 172.21.15.40:40742) 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 -- 172.21.15.40:0/2740519989 learned_addr learned my addr 172.21.15.40:0/2740519989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c1610640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 0x7fb5bc13c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5c0e0f640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 0x7fb5bc13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 -- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 msgr2=0x7fb5bc13c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 0x7fb5bc13c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 -- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 msgr2=0x7fb5bc13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 0x7fb5bc13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.228+0000 7fb5bbfff640 1 -- 172.21.15.40:0/2740519989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc13e960 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c0e0f640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 0x7fb5bc13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c1610640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 0x7fb5bc13c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5bbfff640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00ae040 tx=0x7fb5b0018f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b9070 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b00b4400 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc142b60 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b0040 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc13d750 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc068d00 con 0x7fb5bc0b6170 2024-04-02T16:12:30.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fb5b00bc070 con 0x7fb5bc0b6170 2024-04-02T16:12:30.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 0x7fb5a806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fb5b011fbc0 con 0x7fb5bc0b6170 2024-04-02T16:12:30.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c0e0f640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 0x7fb5a806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.232+0000 7fb5c0e0f640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 0x7fb5a806a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00a850 tx=0x7fb5ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.236+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fb5bc142b60 con 0x7fb5bc0b6170 2024-04-02T16:12:30.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.436+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5bc0b9ae0 con 0x7fb5bc0b6170 2024-04-02T16:12:30.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5b97fa640 1 -- 172.21.15.40:0/2740519989 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7fb5bc13e960 con 0x7fb5bc0b6170 2024-04-02T16:12:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:30.440 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 msgr2=0x7fb5a806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 0x7fb5a806a840 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00a850 tx=0x7fb5ac008040 comp rx=0 tx=0).stop 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 msgr2=0x7fb5bc13e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00ae040 tx=0x7fb5b0018f80 comp rx=0 tx=0).stop 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 shutdown_connections 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5bc1493c0 0x7fb5bc13c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fb5a8068380 0x7fb5a806a840 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5bc0b6170 0x7fb5bc13e1e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 --2- 172.21.15.40:0/2740519989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5bc0b4f00 0x7fb5bc13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 >> 172.21.15.40:0/2740519989 conn(0x7fb5bc02c5d0 msgr2=0x7fb5bc02c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 shutdown_connections 2024-04-02T16:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.440+0000 7fb5c3073640 1 -- 172.21.15.40:0/2740519989 wait complete. 2024-04-02T16:12:30.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.576+0000 7f81561d3640 1 Processor -- start 2024-04-02T16:12:30.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.576+0000 7f81561d3640 1 -- start start 2024-04-02T16:12:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8150073230 0x7f81500b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 0x7f8150149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81500b19e0 con 0x7f81500b23d0 2024-04-02T16:12:30.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f81501497c0 con 0x7f8150073230 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f81500b05c0 con 0x7f8150148e50 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8150073230 0x7f81500b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8150073230 0x7f81500b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40758/0 (socket says 172.21.15.40:40758) 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 -- 172.21.15.40:0/2687921466 learned_addr learned my addr 172.21.15.40:0/2687921466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814ffff640 1 --2- 172.21.15.40:0/2687921466 >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 0x7f8150149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 -- 172.21.15.40:0/2687921466 >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 msgr2=0x7f8150149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 --2- 172.21.15.40:0/2687921466 >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 0x7f8150149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 -- 172.21.15.40:0/2687921466 >> v1:172.21.15.204:6789/0 conn(0x7f81500b23d0 legacy=0x7f815009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 -- 172.21.15.40:0/2687921466 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81500b08a0 con 0x7f8150073230 2024-04-02T16:12:30.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814ffff640 1 --2- 172.21.15.40:0/2687921466 >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 0x7f8150149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814effd640 1 --2- 172.21.15.40:0/2687921466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150073230 0x7f81500b27b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8150146560 tx=0x7f813801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9c1710e2f0ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814e7fc640 1 -- 172.21.15.40:0/2687921466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380a7070 con 0x7f8150073230 2024-04-02T16:12:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814e7fc640 1 -- 172.21.15.40:0/2687921466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81380b0070 con 0x7f8150073230 2024-04-02T16:12:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f814e7fc640 1 -- 172.21.15.40:0/2687921466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380ab890 con 0x7f8150073230 2024-04-02T16:12:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- 172.21.15.40:0/2687921466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150073230 msgr2=0x7f81500b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 --2- 172.21.15.40:0/2687921466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150073230 0x7f81500b27b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8150146560 tx=0x7f813801c450 comp rx=0 tx=0).stop 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- 172.21.15.40:0/2687921466 shutdown_connections 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 --2- 172.21.15.40:0/2687921466 >> v2:172.21.15.204:3300/0 conn(0x7f8150148e50 0x7f8150149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 --2- 172.21.15.40:0/2687921466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150073230 0x7f81500b27b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- 172.21.15.40:0/2687921466 >> 172.21.15.40:0/2687921466 conn(0x7f81500b8fd0 msgr2=0x7f81500b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- 172.21.15.40:0/2687921466 shutdown_connections 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.580+0000 7f81561d3640 1 -- 172.21.15.40:0/2687921466 wait complete. 2024-04-02T16:12:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 Processor -- start 2024-04-02T16:12:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- start start 2024-04-02T16:12:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8150073230 0x7f81501d7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 0x7f81501de530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8150152810 con 0x7f8150148e50 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8150152690 con 0x7f81500b23d0 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8150152990 con 0x7f8150073230 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40760/0 (socket says 172.21.15.40:40760) 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 -- 172.21.15.40:0/779421933 learned_addr learned my addr 172.21.15.40:0/779421933 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814effd640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 0x7f81501de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 -- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8150073230 msgr2=0x7f81501d7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814f7fe640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8150073230 0x7f81501d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8150073230 0x7f81501d7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 -- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 msgr2=0x7f81501de530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 0x7f81501de530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 -- 172.21.15.40:0/779421933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81501d7f20 con 0x7f8150148e50 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814effd640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 0x7f81501de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f814ffff640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f814800c820 tx=0x7f814800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148015020 con 0x7f8150148e50 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8148004240 con 0x7f8150148e50 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81501d8200 con 0x7f8150148e50 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148003620 con 0x7f8150148e50 2024-04-02T16:12:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501df010 con 0x7f8150148e50 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.584+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81501d8530 con 0x7f8150148e50 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.588+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8148018020 con 0x7f8150148e50 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.588+0000 7f8147fff640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 0x7f813006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.588+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8148086a00 con 0x7f8150148e50 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.588+0000 7f814f7fe640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 0x7f813006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.588+0000 7f814f7fe640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 0x7f813006a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8140004450 tx=0x7f8140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.592+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f814801f020 con 0x7f8150148e50 2024-04-02T16:12:30.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.756+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81500685f0 con 0x7f8150148e50 2024-04-02T16:12:30.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f8147fff640 1 -- 172.21.15.40:0/779421933 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16621 (secure 0 0 0) 0x7f8148057050 con 0x7f8150148e50 2024-04-02T16:12:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:30.761 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:16.692754+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 msgr2=0x7f813006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 0x7f813006a840 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8140004450 tx=0x7f8140009040 comp rx=0 tx=0).stop 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 msgr2=0x7f815013ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f814800c820 tx=0x7f814800cce0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 shutdown_connections 2024-04-02T16:12:30.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8130068380 0x7f813006a840 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8150148e50 0x7f815013ce40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81500b23d0 0x7f81501de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 --2- 172.21.15.40:0/779421933 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8150073230 0x7f81501d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 >> 172.21.15.40:0/779421933 conn(0x7f81500b8fd0 msgr2=0x7f81500b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 shutdown_connections 2024-04-02T16:12:30.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.760+0000 7f81561d3640 1 -- 172.21.15.40:0/779421933 wait complete. 2024-04-02T16:12:30.779 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T16:11:19.895393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T16:12:13.528574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T16:12:14.023094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T16:12:30.780 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T16:12:30.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 Processor -- start 2024-04-02T16:12:30.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 -- start start 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd408148370 0x7fd4080b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 0x7fd408152470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40806d120 con 0x7fd4080b7510 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd408148750 con 0x7fd408148370 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd40de87640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd40806d820 con 0x7fd408150080 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd4077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd408148370 0x7fd4080b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd4077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd408148370 0x7fd4080b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40766/0 (socket says 172.21.15.40:40766) 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd4077fe640 1 -- 172.21.15.40:0/3538557254 learned_addr learned my addr 172.21.15.40:0/3538557254 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd406ffd640 1 --1- 172.21.15.40:0/3538557254 >> v1:172.21.15.40:6789/0 conn(0x7fd4080b7510 0x7fd40806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54166/0 (socket says 172.21.15.40:54166) 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd407fff640 1 --2- 172.21.15.40:0/3538557254 >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 0x7fd408152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd4077fe640 1 -- 172.21.15.40:0/3538557254 >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 msgr2=0x7fd408152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.896+0000 7fd4077fe640 1 --2- 172.21.15.40:0/3538557254 >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 0x7fd408152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4077fe640 1 -- 172.21.15.40:0/3538557254 >> v1:172.21.15.40:6789/0 conn(0x7fd4080b7510 legacy=0x7fd40806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4077fe640 1 -- 172.21.15.40:0/3538557254 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd408147770 con 0x7fd408148370 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4067fc640 1 -- 172.21.15.40:0/3538557254 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001158280 0 0) 0x7fd40806d120 con 0x7fd4080b7510 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd407fff640 1 --2- 172.21.15.40:0/3538557254 >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 0x7fd408152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4077fe640 1 --2- 172.21.15.40:0/3538557254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4080b81c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd408070010 tx=0x7fd3f801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d968a7eae6313a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4067fc640 1 -- 172.21.15.40:0/3538557254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80b0070 con 0x7fd408148370 2024-04-02T16:12:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4067fc640 1 -- 172.21.15.40:0/3538557254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f80ab600 con 0x7fd408148370 2024-04-02T16:12:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd4067fc640 1 -- 172.21.15.40:0/3538557254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80ab9d0 con 0x7fd408148370 2024-04-02T16:12:30.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- 172.21.15.40:0/3538557254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 msgr2=0x7fd4080b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 --2- 172.21.15.40:0/3538557254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4080b81c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd408070010 tx=0x7fd3f801c400 comp rx=0 tx=0).stop 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- 172.21.15.40:0/3538557254 shutdown_connections 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 --2- 172.21.15.40:0/3538557254 >> v2:172.21.15.204:3300/0 conn(0x7fd408150080 0x7fd408152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 --2- 172.21.15.40:0/3538557254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4080b81c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- 172.21.15.40:0/3538557254 >> 172.21.15.40:0/3538557254 conn(0x7fd4080bb640 msgr2=0x7fd4080b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- 172.21.15.40:0/3538557254 shutdown_connections 2024-04-02T16:12:30.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- 172.21.15.40:0/3538557254 wait complete. 2024-04-02T16:12:30.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 Processor -- start 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.900+0000 7fd40de87640 1 -- start start 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 0x7fd4081430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd408150080 0x7fd4080b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd4080b8700 con 0x7fd408148370 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd40806e3a0 con 0x7fd4080b7510 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4080b8880 con 0x7fd408150080 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40778/0 (socket says 172.21.15.40:40778) 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 -- 172.21.15.40:0/2129918826 learned_addr learned my addr 172.21.15.40:0/2129918826 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd4077fe640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 0x7fd4081430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 -- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd408150080 msgr2=0x7fd4080b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd407fff640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd408150080 0x7fd4080b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd408150080 0x7fd4080b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 -- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 msgr2=0x7fd4081430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 0x7fd4081430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 -- 172.21.15.40:0/2129918826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4080b54c0 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd4077fe640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 0x7fd4081430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd406ffd640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c520 tx=0x7fd3fc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc018070 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0074e0 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4080b51e0 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0049f0 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4080b4300 con 0x7fd408148370 2024-04-02T16:12:30.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40814fd90 con 0x7fd408148370 2024-04-02T16:12:30.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd3fc01b070 con 0x7fd408148370 2024-04-02T16:12:30.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 0x7fd3f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:30.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd3fc085d60 con 0x7fd408148370 2024-04-02T16:12:30.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.904+0000 7fd4077fe640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 0x7fd3f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:30.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.908+0000 7fd4077fe640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 0x7fd3f406a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:30.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:30.908+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd3fc0563b0 con 0x7fd408148370 2024-04-02T16:12:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.072+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd408068d00 con 0x7fd408148370 2024-04-02T16:12:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.072+0000 7fd3effff640 1 -- 172.21.15.40:0/2129918826 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fd3fc055cb0 con 0x7fd408148370 2024-04-02T16:12:31.075 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 msgr2=0x7fd3f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 0x7fd3f406a840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80ad000 tx=0x7fd3f8008040 comp rx=0 tx=0).stop 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 msgr2=0x7fd4081e3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c520 tx=0x7fd3fc00c9e0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 shutdown_connections 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd408150080 0x7fd4080b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd3f4068380 0x7fd3f406a840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd408148370 0x7fd4081e3dc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 --2- 172.21.15.40:0/2129918826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4080b7510 0x7fd4081430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 >> 172.21.15.40:0/2129918826 conn(0x7fd4080bb640 msgr2=0x7fd4080719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 shutdown_connections 2024-04-02T16:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.076+0000 7fd40de87640 1 -- 172.21.15.40:0/2129918826 wait complete. 2024-04-02T16:12:31.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T16:12:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 Processor -- start 2024-04-02T16:12:31.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 -- start start 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f16980b8420 0x7f16980b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16980b7020 0x7f16980b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16980b8800 con 0x7f16980b1810 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f16980b59b0 con 0x7f16980b7020 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f169decd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f16980b74b0 con 0x7f16980b8420 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16980b7020 0x7f16980b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16980b7020 0x7f16980b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40786/0 (socket says 172.21.15.40:40786) 2024-04-02T16:12:31.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 -- 172.21.15.40:0/3151515083 learned_addr learned my addr 172.21.15.40:0/3151515083 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f16977fe640 1 --2- 172.21.15.40:0/3151515083 >> v2:172.21.15.204:3300/0 conn(0x7f16980b8420 0x7f16980b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 -- 172.21.15.40:0/3151515083 >> v2:172.21.15.204:3300/0 conn(0x7f16980b8420 msgr2=0x7f16980b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 --2- 172.21.15.40:0/3151515083 >> v2:172.21.15.204:3300/0 conn(0x7f16980b8420 0x7f16980b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 -- 172.21.15.40:0/3151515083 >> v1:172.21.15.204:6789/0 conn(0x7f16980b1810 legacy=0x7f16980719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.220+0000 7f1696ffd640 1 -- 172.21.15.40:0/3151515083 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16980b5c00 con 0x7f16980b7020 2024-04-02T16:12:31.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f1696ffd640 1 --2- 172.21.15.40:0/3151515083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16980b7020 0x7f16980b4a10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16980b5b30 tx=0x7f168c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1da2ff42f9f0fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f16967fc640 1 -- 172.21.15.40:0/3151515083 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0b0070 con 0x7f16980b7020 2024-04-02T16:12:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f16967fc640 1 -- 172.21.15.40:0/3151515083 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c0ab600 con 0x7f16980b7020 2024-04-02T16:12:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f16967fc640 1 -- 172.21.15.40:0/3151515083 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c0ab9d0 con 0x7f16980b7020 2024-04-02T16:12:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- 172.21.15.40:0/3151515083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16980b7020 msgr2=0x7f16980b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- 172.21.15.40:0/3151515083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16980b7020 0x7f16980b4a10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16980b5b30 tx=0x7f168c01c380 comp rx=0 tx=0).stop 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- 172.21.15.40:0/3151515083 shutdown_connections 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- 172.21.15.40:0/3151515083 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16980b7020 0x7f16980b4a10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- 172.21.15.40:0/3151515083 >> v2:172.21.15.204:3300/0 conn(0x7f16980b8420 0x7f16980b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- 172.21.15.40:0/3151515083 >> 172.21.15.40:0/3151515083 conn(0x7f16980bb640 msgr2=0x7f16980b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- 172.21.15.40:0/3151515083 shutdown_connections 2024-04-02T16:12:31.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- 172.21.15.40:0/3151515083 wait complete. 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 Processor -- start 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- start start 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 0x7f1698141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16980b8420 0x7f1698147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1698152e40 con 0x7f1698150600 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1698152fc0 con 0x7f16980b1810 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f169decd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1698153140 con 0x7f16980b8420 2024-04-02T16:12:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f1697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f1697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40788/0 (socket says 172.21.15.40:40788) 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.224+0000 7f1697fff640 1 -- 172.21.15.40:0/366264691 learned_addr learned my addr 172.21.15.40:0/366264691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1696ffd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16980b8420 0x7f1698147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f16977fe640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 0x7f1698141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 -- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16980b8420 msgr2=0x7f1698147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16980b8420 0x7f1698147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 -- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 msgr2=0x7f1698141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 0x7f1698141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 -- 172.21.15.40:0/366264691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1698148d20 con 0x7f1698150600 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f16977fe640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 0x7f1698141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f1697fff640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f168800c3f0 tx=0x7f168800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688011790 con 0x7f1698150600 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1688017070 con 0x7f1698150600 2024-04-02T16:12:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688011b70 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1698150b50 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1698146800 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1698068d00 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1688011d10 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 0x7f168406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f1688086740 con 0x7f1698150600 2024-04-02T16:12:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.228+0000 7f16977fe640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 0x7f168406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.232+0000 7f16977fe640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 0x7f168406a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1690009f50 tx=0x7f169000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.232+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f1688056d90 con 0x7f1698150600 2024-04-02T16:12:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.392+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f16980b2bf0 con 0x7f1698150600 2024-04-02T16:12:31.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.392+0000 7f167ffff640 1 -- 172.21.15.40:0/366264691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f1688056690 con 0x7f1698150600 2024-04-02T16:12:31.394 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 msgr2=0x7f168406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 0x7f168406a870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1690009f50 tx=0x7f169000b040 comp rx=0 tx=0).stop 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 msgr2=0x7f1698147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f168800c3f0 tx=0x7f168800c8b0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 shutdown_connections 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16980b8420 0x7f1698147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f16840683b0 0x7f168406a870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1698150600 0x7f1698147ac0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 --2- 172.21.15.40:0/366264691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16980b1810 0x7f1698141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 >> 172.21.15.40:0/366264691 conn(0x7f16980bb640 msgr2=0x7f16980b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 shutdown_connections 2024-04-02T16:12:31.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.396+0000 7f169decd640 1 -- 172.21.15.40:0/366264691 wait complete. 2024-04-02T16:12:31.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T16:12:31.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 Processor -- start 2024-04-02T16:12:31.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 -- start start 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49400b23d0 0x7f49400b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4940073230 0x7f494009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49400b19e0 con 0x7f494009dfe0 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f49400b05c0 con 0x7f49400b23d0 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4946380640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49400b08a0 con 0x7f4940073230 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49400b23d0 0x7f49400b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49400b23d0 0x7f49400b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40798/0 (socket says 172.21.15.40:40798) 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 -- 172.21.15.40:0/2264629007 learned_addr learned my addr 172.21.15.40:0/2264629007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f4937fff640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4940073230 0x7f494009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 -- 172.21.15.40:0/2264629007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4940073230 msgr2=0x7f494009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4940073230 0x7f494009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 -- 172.21.15.40:0/2264629007 >> v1:172.21.15.204:6789/0 conn(0x7f494009dfe0 legacy=0x7f4940148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.536+0000 7f493ffff640 1 -- 172.21.15.40:0/2264629007 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49400b0b80 con 0x7f49400b23d0 2024-04-02T16:12:31.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f493ffff640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49400b23d0 0x7f49400b27b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4930009870 tx=0x7f493001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54abdf5eecfd8be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f493f7fe640 1 -- 172.21.15.40:0/2264629007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300ab5f0 con 0x7f49400b23d0 2024-04-02T16:12:31.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f493f7fe640 1 -- 172.21.15.40:0/2264629007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49300b0070 con 0x7f49400b23d0 2024-04-02T16:12:31.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f493f7fe640 1 -- 172.21.15.40:0/2264629007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300ab9b0 con 0x7f49400b23d0 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- 172.21.15.40:0/2264629007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49400b23d0 msgr2=0x7f49400b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49400b23d0 0x7f49400b27b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4930009870 tx=0x7f493001c4d0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- 172.21.15.40:0/2264629007 shutdown_connections 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4940073230 0x7f494009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 --2- 172.21.15.40:0/2264629007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49400b23d0 0x7f49400b27b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- 172.21.15.40:0/2264629007 >> 172.21.15.40:0/2264629007 conn(0x7f49400b8fd0 msgr2=0x7f49400b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- 172.21.15.40:0/2264629007 shutdown_connections 2024-04-02T16:12:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- 172.21.15.40:0/2264629007 wait complete. 2024-04-02T16:12:31.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 Processor -- start 2024-04-02T16:12:31.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.540+0000 7f4946380640 1 -- start start 2024-04-02T16:12:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f494009dfe0 0x7f49400b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 0x7f4940137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4940152a20 con 0x7f4940073230 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49401528a0 con 0x7f494009dfe0 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4940152ba0 con 0x7f49400b23d0 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40812/0 (socket says 172.21.15.40:40812) 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 -- 172.21.15.40:0/423517935 learned_addr learned my addr 172.21.15.40:0/423517935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4937fff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f494009dfe0 0x7f49400b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f494491d640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 0x7f4940137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 -- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 msgr2=0x7f4940137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 0x7f4940137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 -- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f494009dfe0 msgr2=0x7f49400b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f494009dfe0 0x7f49400b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 -- 172.21.15.40:0/423517935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494013e6c0 con 0x7f4940073230 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f494491d640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 0x7f4940137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493ffff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49300ad040 tx=0x7f493001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300b4800 con 0x7f4940073230 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49300b49a0 con 0x7f4940073230 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300b4d90 con 0x7f4940073230 2024-04-02T16:12:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494013e9a0 con 0x7f4940073230 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f494013ceb0 con 0x7f4940073230 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49400685f0 con 0x7f4940073230 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f49300c7440 con 0x7f4940073230 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 0x7f492006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f4930120fb0 con 0x7f4940073230 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4937fff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 0x7f492006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.544+0000 7f4937fff640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 0x7f492006a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4940137e30 tx=0x7f4928008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.548+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f4930016150 con 0x7f4940073230 2024-04-02T16:12:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.712+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f49400b1690 con 0x7f4940073230 2024-04-02T16:12:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.712+0000 7f493d7fa640 1 -- 172.21.15.40:0/423517935 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f49300abe20 con 0x7f4940073230 2024-04-02T16:12:31.713 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-04-02T16:12:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 msgr2=0x7f492006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 0x7f492006a870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4940137e30 tx=0x7f4928008040 comp rx=0 tx=0).stop 2024-04-02T16:12:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 msgr2=0x7f49400ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49300ad040 tx=0x7f493001ad10 comp rx=0 tx=0).stop 2024-04-02T16:12:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 shutdown_connections 2024-04-02T16:12:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49400b23d0 0x7f4940137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49200683b0 0x7f492006a870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f494009dfe0 0x7f49400b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 --2- 172.21.15.40:0/423517935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4940073230 0x7f49400ae5b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 >> 172.21.15.40:0/423517935 conn(0x7f49400b8fd0 msgr2=0x7f49400b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 shutdown_connections 2024-04-02T16:12:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.716+0000 7f4946380640 1 -- 172.21.15.40:0/423517935 wait complete. 2024-04-02T16:12:31.734 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:31.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:12:31.780 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:12:31.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.856+0000 7f49c5094640 1 Processor -- start 2024-04-02T16:12:31.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.856+0000 7f49c5094640 1 -- start start 2024-04-02T16:12:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49c5094640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49c00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49c5094640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49c0148e50 0x7f49c01492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49c5094640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c00b19e0 con 0x7f49c0073230 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49c5094640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f49c0149800 con 0x7f49c0148e50 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49c5094640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49c00b05c0 con 0x7f49c00b23d0 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49be59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49c0073230 0x7f49c009daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54170/0 (socket says 172.21.15.40:54170) 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49be59c640 1 -- 172.21.15.40:0/313792554 learned_addr learned my addr 172.21.15.40:0/313792554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49c00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bf59e640 1 --2- 172.21.15.40:0/313792554 >> v2:172.21.15.204:3300/0 conn(0x7f49c0148e50 0x7f49c01492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405651782 0 0) 0x7f49c00b19e0 con 0x7f49c0073230 2024-04-02T16:12:31.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49ac003530 con 0x7f49c0073230 2024-04-02T16:12:31.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761262481 0 0) 0x7f49ac003530 con 0x7f49c0073230 2024-04-02T16:12:31.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 msgr2=0x7f49c00b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49c00b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 >> v2:172.21.15.204:3300/0 conn(0x7f49c0148e50 msgr2=0x7f49c01492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 --2- 172.21.15.40:0/313792554 >> v2:172.21.15.204:3300/0 conn(0x7f49c0148e50 0x7f49c01492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c00b08a0 con 0x7f49c0073230 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f49a8001a00 con 0x7f49c0073230 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 >> v1:172.21.15.40:6789/0 conn(0x7f49c0073230 legacy=0x7f49c009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49ac004d50 0x7f49ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0149800 con 0x7f49ac0039c0 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00b05c0 con 0x7f49ac004d50 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac00fde0 con 0x7f49c00b23d0 2024-04-02T16:12:31.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49be59c640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bf59e640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49ac004d50 0x7f49ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 -- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 msgr2=0x7f49ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b23d0 0x7f49ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 -- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49ac004d50 msgr2=0x7f49ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49ac004d50 0x7f49ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 -- 172.21.15.40:0/313792554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac003530 con 0x7f49ac0039c0 2024-04-02T16:12:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.860+0000 7f49bed9d640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49ac003de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009ae0 tx=0x7f49b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28f0d8e514c9b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00b2070 con 0x7f49ac0039c0 2024-04-02T16:12:31.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b00a5550 con 0x7f49ac0039c0 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bdd9b640 1 -- 172.21.15.40:0/313792554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00096b0 con 0x7f49ac0039c0 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/313792554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 msgr2=0x7f49ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 --2- 172.21.15.40:0/313792554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49ac003de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009ae0 tx=0x7f49b001ca70 comp rx=0 tx=0).stop 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bf59e640 1 -- 172.21.15.40:0/313792554 reap_dead start 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/313792554 shutdown_connections 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/313792554 >> 172.21.15.40:0/313792554 conn(0x7f49c00b8fd0 msgr2=0x7f49c00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/313792554 shutdown_connections 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/313792554 wait complete. 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 Processor -- start 2024-04-02T16:12:31.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- start start 2024-04-02T16:12:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c009d450 0x7f49c01dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 0x7f49c013f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00ae7b0 con 0x7f49ac0039c0 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49c00ae630 con 0x7f49c01dce50 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f49c00ae930 con 0x7f49c009d450 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40818/0 (socket says 172.21.15.40:40818) 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 -- 172.21.15.40:0/2339781695 learned_addr learned my addr 172.21.15.40:0/2339781695 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49be59c640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c009d450 0x7f49c01dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bf59e640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 0x7f49c013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 -- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c009d450 msgr2=0x7f49c01dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c009d450 0x7f49c01dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 -- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 msgr2=0x7f49c013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 0x7f49c013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 -- 172.21.15.40:0/2339781695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c0146270 con 0x7f49ac0039c0 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bf59e640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 0x7f49c013f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:31.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49bed9d640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f49b00b0040 tx=0x7f49ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00bc030 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b000f000 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c009d180 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b00b2040 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.864+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01400d0 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c00685f0 con 0x7f49ac0039c0 2024-04-02T16:12:31.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f49b00bf070 con 0x7f49ac0039c0 2024-04-02T16:12:31.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49b77fe640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 0x7f49a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:31.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f49b0120440 con 0x7f49ac0039c0 2024-04-02T16:12:31.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49be59c640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 0x7f49a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:31.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.868+0000 7f49be59c640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 0x7f49a006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49c009ccd0 tx=0x7f49a8002120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:31.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:31.872+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f49b0016780 con 0x7f49ac0039c0 2024-04-02T16:12:32.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c00b1690 con 0x7f49ac0039c0 2024-04-02T16:12:32.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49b77fe640 1 -- 172.21.15.40:0/2339781695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7f49b00f0a90 con 0x7f49ac0039c0 2024-04-02T16:12:32.077 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:32.077 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:32.077 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:32.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 msgr2=0x7f49a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 0x7f49a006a840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f49c009ccd0 tx=0x7f49a8002120 comp rx=0 tx=0).stop 2024-04-02T16:12:32.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 msgr2=0x7f49c009b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f49b00b0040 tx=0x7f49ac005840 comp rx=0 tx=0).stop 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 shutdown_connections 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c009d450 0x7f49c01dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49a0068380 0x7f49a006a840 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c01dce50 0x7f49c013f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 --2- 172.21.15.40:0/2339781695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49ac0039c0 0x7f49c009b5d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 >> 172.21.15.40:0/2339781695 conn(0x7f49c00b8fd0 msgr2=0x7f49c006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 shutdown_connections 2024-04-02T16:12:32.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.076+0000 7f49c5094640 1 -- 172.21.15.40:0/2339781695 wait complete. 2024-04-02T16:12:32.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 Processor -- start 2024-04-02T16:12:32.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 -- start start 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f68280b5860 0x7f6828141b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68280bb850 con 0x7f682806e070 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68280b6300 con 0x7f68280bb470 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682eb9e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f682806d0d0 con 0x7f68280b5860 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f682806e070 0x7f6828141440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54178/0 (socket says 172.21.15.40:54178) 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 -- 172.21.15.40:0/983209698 learned_addr learned my addr 172.21.15.40:0/983209698 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682d13b640 1 --2- 172.21.15.40:0/983209698 >> v2:172.21.15.204:3300/0 conn(0x7f68280b5860 0x7f6828141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52662175 0 0) 0x7f68280bb850 con 0x7f682806e070 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6810003530 con 0x7f682806e070 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2009142806 0 0) 0x7f6810003530 con 0x7f682806e070 2024-04-02T16:12:32.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 >> v2:172.21.15.204:3300/0 conn(0x7f68280b5860 msgr2=0x7f6828141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 --2- 172.21.15.40:0/983209698 >> v2:172.21.15.204:3300/0 conn(0x7f68280b5860 0x7f6828141b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 >> v1:172.21.15.204:6789/0 conn(0x7f68280bb470 legacy=0x7f682806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68280b6b90 con 0x7f682806e070 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f681c001680 con 0x7f682806e070 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 >> v1:172.21.15.40:6789/0 conn(0x7f682806e070 legacy=0x7f6828141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f682806d0d0 con 0x7f6810004810 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f681000c170 con 0x7f68100051d0 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f681000c370 con 0x7f68100039c0 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682c93a640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682d13b640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 -- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68100039c0 msgr2=0x7f6810003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 -- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68100051d0 msgr2=0x7f681000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f6827fff640 1 -- 172.21.15.40:0/983209698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6810003530 con 0x7f6810004810 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682c93a640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68100039c0 0x7f6810003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682d13b640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68100051d0 0x7f681000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:32.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.264+0000 7f682d13b640 1 -- 172.21.15.40:0/983209698 reap_dead start 2024-04-02T16:12:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 0x7f6810004c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c210 tx=0x7f681c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55da0477618461b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c007d20 con 0x7f6810004810 2024-04-02T16:12:32.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681c0a9070 con 0x7f6810004810 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f68277fe640 1 -- 172.21.15.40:0/983209698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c005da0 con 0x7f6810004810 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 msgr2=0x7f6810004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 0x7f6810004c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c210 tx=0x7f681c01cc20 comp rx=0 tx=0).stop 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- 172.21.15.40:0/983209698 shutdown_connections 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 --2- 172.21.15.40:0/983209698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6810004810 0x7f6810004c90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- 172.21.15.40:0/983209698 >> 172.21.15.40:0/983209698 conn(0x7f682802c5d0 msgr2=0x7f682802c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- 172.21.15.40:0/983209698 shutdown_connections 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- 172.21.15.40:0/983209698 wait complete. 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 Processor -- start 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- start start 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6810004810 0x7f68281e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 0x7f68280b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f682806def0 con 0x7f6828149090 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f682806dd70 con 0x7f682814a9e0 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682eb9e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6828141c50 con 0x7f6810004810 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40836/0 (socket says 172.21.15.40:40836) 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 -- 172.21.15.40:0/2906262779 learned_addr learned my addr 172.21.15.40:0/2906262779 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682c93a640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6810004810 0x7f68281e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682d13b640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 0x7f68280b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 -- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6810004810 msgr2=0x7f68281e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6810004810 0x7f68281e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 -- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 msgr2=0x7f68280b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 0x7f68280b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f6827fff640 1 -- 172.21.15.40:0/2906262779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68281e0f40 con 0x7f6828149090 2024-04-02T16:12:32.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.268+0000 7f682d13b640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 0x7f68280b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f6827fff640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f681c01ca50 tx=0x7f6810005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c0bc070 con 0x7f6828149090 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681c000ee0 con 0x7f6828149090 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682814b960 con 0x7f6828149090 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c007d20 con 0x7f6828149090 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281e04e0 con 0x7f6828149090 2024-04-02T16:12:32.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68281e0010 con 0x7f6828149090 2024-04-02T16:12:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f681c0bf070 con 0x7f6828149090 2024-04-02T16:12:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 0x7f681406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f681c11fd00 con 0x7f6828149090 2024-04-02T16:12:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f682c93a640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 0x7f681406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.272+0000 7f682c93a640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 0x7f681406a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6818000c10 tx=0x7f6818009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.276+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f681c016970 con 0x7f6828149090 2024-04-02T16:12:32.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.440+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68280bb850 con 0x7f6828149090 2024-04-02T16:12:32.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.440+0000 7f68257fa640 1 -- 172.21.15.40:0/2906262779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16621 (secure 0 0 0) 0x7f681c0f02b0 con 0x7f6828149090 2024-04-02T16:12:32.443 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:32.443 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:16.692754+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 msgr2=0x7f681406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 0x7f681406a840 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6818000c10 tx=0x7f6818009040 comp rx=0 tx=0).stop 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 msgr2=0x7f682814a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f681c01ca50 tx=0x7f6810005840 comp rx=0 tx=0).stop 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 shutdown_connections 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f6814068380 0x7f681406a840 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f682814a9e0 0x7f68280b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6828149090 0x7f682814a4a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 --2- 172.21.15.40:0/2906262779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6810004810 0x7f68281e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 >> 172.21.15.40:0/2906262779 conn(0x7f682802c5d0 msgr2=0x7f6828071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 shutdown_connections 2024-04-02T16:12:32.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.444+0000 7f682eb9e640 1 -- 172.21.15.40:0/2906262779 wait complete. 2024-04-02T16:12:32.463 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T16:12:32.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:32.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 Processor -- start 2024-04-02T16:12:32.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 -- start start 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b6170 0x7eff30148e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff301493c0 0x7eff3014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff300b52e0 con 0x7eff300b4f00 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff300b6f00 con 0x7eff301493c0 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff3497c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff300b71a0 con 0x7eff300b6170 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff301493c0 0x7eff3014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7eff301493c0 0x7eff3014fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40840/0 (socket says 172.21.15.40:40840) 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 -- 172.21.15.40:0/3024073722 learned_addr learned my addr 172.21.15.40:0/3024073722 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2e59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff300b4f00 0x7eff300bb300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54190/0 (socket says 172.21.15.40:54190) 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2dd9b640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b6170 0x7eff30148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 -- 172.21.15.40:0/3024073722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b6170 msgr2=0x7eff30148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b6170 0x7eff30148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 -- 172.21.15.40:0/3024073722 >> v1:172.21.15.40:6789/0 conn(0x7eff300b4f00 legacy=0x7eff300bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2ed9d640 1 -- 172.21.15.40:0/3024073722 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff301497d0 con 0x7eff301493c0 2024-04-02T16:12:32.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.580+0000 7eff2d59a640 1 -- 172.21.15.40:0/3024073722 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225162508 0 0) 0x7eff20001120 con 0x7eff300b4f00 2024-04-02T16:12:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2ed9d640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff301493c0 0x7eff3014fe40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff28009870 tx=0x7eff2801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ea9ef09d2953d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2d59a640 1 -- 172.21.15.40:0/3024073722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff280b0070 con 0x7eff301493c0 2024-04-02T16:12:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2d59a640 1 -- 172.21.15.40:0/3024073722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff280ab600 con 0x7eff301493c0 2024-04-02T16:12:32.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2d59a640 1 -- 172.21.15.40:0/3024073722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff280ab9d0 con 0x7eff301493c0 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3024073722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff301493c0 msgr2=0x7eff3014fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff301493c0 0x7eff3014fe40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7eff28009870 tx=0x7eff2801c440 comp rx=0 tx=0).stop 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3024073722 shutdown_connections 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff301493c0 0x7eff3014fe40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- 172.21.15.40:0/3024073722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b6170 0x7eff30148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3024073722 >> 172.21.15.40:0/3024073722 conn(0x7eff3002c5d0 msgr2=0x7eff3002c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3024073722 shutdown_connections 2024-04-02T16:12:32.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3024073722 wait complete. 2024-04-02T16:12:32.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 Processor -- start 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- start start 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b4f00 0x7eff301417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff301493c0 0x7eff300b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff30072b90 con 0x7eff300b6170 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff300b5a20 con 0x7eff301493c0 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff30152ba0 con 0x7eff300b4f00 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40854/0 (socket says 172.21.15.40:40854) 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 -- 172.21.15.40:0/3737129673 learned_addr learned my addr 172.21.15.40:0/3737129673 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2ed9d640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff301493c0 0x7eff300b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2e59c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b4f00 0x7eff301417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 -- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b4f00 msgr2=0x7eff301417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b4f00 0x7eff301417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 -- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff301493c0 msgr2=0x7eff300b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff301493c0 0x7eff300b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 -- 172.21.15.40:0/3737129673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff300ba860 con 0x7eff300b6170 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff2dd9b640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7eff18007e40 tx=0x7eff1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18014620 con 0x7eff300b6170 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff18019070 con 0x7eff300b6170 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff30142f60 con 0x7eff300b6170 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18014a00 con 0x7eff300b6170 2024-04-02T16:12:32.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.584+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff30142c60 con 0x7eff300b6170 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff30068d00 con 0x7eff300b6170 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7eff18014ba0 con 0x7eff300b6170 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff26ffd640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 0x7eff1006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7eff18087190 con 0x7eff300b6170 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff2e59c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 0x7eff1006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.588+0000 7eff2e59c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 0x7eff1006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f70 tx=0x7eff20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.592+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7eff180577e0 con 0x7eff300b6170 2024-04-02T16:12:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff3006bde0 con 0x7eff300b6170 2024-04-02T16:12:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff26ffd640 1 -- 172.21.15.40:0/3737129673 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7eff180570e0 con 0x7eff300b6170 2024-04-02T16:12:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:32.789 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:32.789 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 msgr2=0x7eff1006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 0x7eff1006a840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff20000f70 tx=0x7eff20009040 comp rx=0 tx=0).stop 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 msgr2=0x7eff30142720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7eff18007e40 tx=0x7eff1800e4b0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 shutdown_connections 2024-04-02T16:12:32.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7eff10068380 0x7eff1006a840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff301493c0 0x7eff300b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff300b6170 0x7eff30142720 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 --2- 172.21.15.40:0/3737129673 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff300b4f00 0x7eff301417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 >> 172.21.15.40:0/3737129673 conn(0x7eff3002c5d0 msgr2=0x7eff30145800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 shutdown_connections 2024-04-02T16:12:32.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.788+0000 7eff3497c640 1 -- 172.21.15.40:0/3737129673 wait complete. 2024-04-02T16:12:32.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:12:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.924+0000 7f411e501640 1 Processor -- start 2024-04-02T16:12:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.924+0000 7f411e501640 1 -- start start 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411e501640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 0x7f41180af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411e501640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41180b43a0 0x7f411814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411e501640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41180ae830 con 0x7f41180b3800 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411e501640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f41180b2b50 con 0x7f41180b43a0 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411e501640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f41180b0300 con 0x7f411806c8a0 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41180b43a0 0x7f411814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41180b43a0 0x7f411814f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40870/0 (socket says 172.21.15.40:40870) 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 -- 172.21.15.40:0/2496691132 learned_addr learned my addr 172.21.15.40:0/2496691132 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f4117fff640 1 --2- 172.21.15.40:0/2496691132 >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 0x7f41180af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 -- 172.21.15.40:0/2496691132 >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 msgr2=0x7f41180af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 --2- 172.21.15.40:0/2496691132 >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 0x7f41180af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 -- 172.21.15.40:0/2496691132 >> v1:172.21.15.204:6789/0 conn(0x7f41180b3800 legacy=0x7f41180b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 -- 172.21.15.40:0/2496691132 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411806cc80 con 0x7f41180b43a0 2024-04-02T16:12:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f4117fff640 1 --2- 172.21.15.40:0/2496691132 >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 0x7f41180af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f411ca9e640 1 --2- 172.21.15.40:0/2496691132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411814f0b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f411806e070 tx=0x7f410801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550c93eb055560c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f4116ffd640 1 -- 172.21.15.40:0/2496691132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080b0070 con 0x7f41180b43a0 2024-04-02T16:12:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f4116ffd640 1 -- 172.21.15.40:0/2496691132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41080ab600 con 0x7f41180b43a0 2024-04-02T16:12:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.928+0000 7f4116ffd640 1 -- 172.21.15.40:0/2496691132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080ab9d0 con 0x7f41180b43a0 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/2496691132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 msgr2=0x7f411814f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- 172.21.15.40:0/2496691132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411814f0b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f411806e070 tx=0x7f410801c450 comp rx=0 tx=0).stop 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/2496691132 shutdown_connections 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- 172.21.15.40:0/2496691132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411814f0b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- 172.21.15.40:0/2496691132 >> v2:172.21.15.204:3300/0 conn(0x7f411806c8a0 0x7f41180af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/2496691132 >> 172.21.15.40:0/2496691132 conn(0x7f41180b8fd0 msgr2=0x7f41180b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/2496691132 shutdown_connections 2024-04-02T16:12:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/2496691132 wait complete. 2024-04-02T16:12:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 Processor -- start 2024-04-02T16:12:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- start start 2024-04-02T16:12:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f411806c8a0 0x7f411814bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41180b3800 0x7f411809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f41180b3130 con 0x7f41180b43a0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41180af690 con 0x7f41180b3800 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f411809abd0 con 0x7f411806c8a0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40886/0 (socket says 172.21.15.40:40886) 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 -- 172.21.15.40:0/416442057 learned_addr learned my addr 172.21.15.40:0/416442057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f41177fe640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41180b3800 0x7f411809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f4117fff640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f411806c8a0 0x7f411814bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 -- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f411806c8a0 msgr2=0x7f411814bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f411806c8a0 0x7f411814bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 -- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41180b3800 msgr2=0x7f411809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41180b3800 0x7f411809cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 -- 172.21.15.40:0/416442057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411809c4b0 con 0x7f41180b43a0 2024-04-02T16:12:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411ca9e640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f410801a080 tx=0x7f41080191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080b9070 con 0x7f41180b43a0 2024-04-02T16:12:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41080b4620 con 0x7f41180b43a0 2024-04-02T16:12:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080b0040 con 0x7f41180b43a0 2024-04-02T16:12:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4118148620 con 0x7f41180b43a0 2024-04-02T16:12:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411809b620 con 0x7f41180b43a0 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.932+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41180685f0 con 0x7f41180b43a0 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.936+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f41080bc070 con 0x7f41180b43a0 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.936+0000 7f4114ff9640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 0x7f410406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.936+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f410811fbc0 con 0x7f41180b43a0 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.936+0000 7f4117fff640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 0x7f410406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.936+0000 7f4117fff640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 0x7f410406a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41100075b0 tx=0x7f4110007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:32.940+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f41080160d0 con 0x7f41180b43a0 2024-04-02T16:12:33.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41180b32b0 con 0x7f41180b43a0 2024-04-02T16:12:33.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f4114ff9640 1 -- 172.21.15.40:0/416442057 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+3534 (secure 0 0 0) 0x7f41080f0210 con 0x7f41180b43a0 2024-04-02T16:12:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:33.141 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T16:12:15.068670+0000","modified":"2024-04-02T16:12:17.070715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4989},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4989":{"gid":4989,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/2966573419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2966573419},{"type":"v1","addr":"172.21.15.40:6833","nonce":2966573419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4989,"qdb_cluster":[4989]},"id":2}]} 2024-04-02T16:12:33.141 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 msgr2=0x7f410406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 0x7f410406a840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f41100075b0 tx=0x7f4110007540 comp rx=0 tx=0).stop 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 msgr2=0x7f411809b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f410801a080 tx=0x7f41080191f0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 shutdown_connections 2024-04-02T16:12:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4104068380 0x7f410406a840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41180b43a0 0x7f411809b9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41180b3800 0x7f411809cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 --2- 172.21.15.40:0/416442057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f411806c8a0 0x7f411814bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 >> 172.21.15.40:0/416442057 conn(0x7f41180b8fd0 msgr2=0x7f411806ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 shutdown_connections 2024-04-02T16:12:33.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.140+0000 7f411e501640 1 -- 172.21.15.40:0/416442057 wait complete. 2024-04-02T16:12:33.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:33.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 Processor -- start 2024-04-02T16:12:33.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 -- start start 2024-04-02T16:12:33.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80b3500 0x7f53f814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f80b6310 con 0x7f53f814b390 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f53f80b55d0 con 0x7f53f80b3500 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53fc992640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f53f80b3fa0 con 0x7f53f80b4bf0 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53f5d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80b3500 0x7f53f814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53f659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53f6d9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f53f814b390 0x7f53f80b8c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:44800/0 (socket says 172.21.15.40:44800) 2024-04-02T16:12:33.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53f6d9d640 1 -- 172.21.15.40:0/3183585744 learned_addr learned my addr 172.21.15.40:0/3183585744 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:33.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.276+0000 7f53f5d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f53f80b3500 0x7f53f814ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:49698/0 (socket says 172.21.15.40:49698) 2024-04-02T16:12:33.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f5d9b640 1 -- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 msgr2=0x7f53f80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:33.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f5d9b640 1 --2- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f80b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f5d9b640 1 -- 172.21.15.40:0/3183585744 >> v1:172.21.15.204:6789/0 conn(0x7f53f814b390 legacy=0x7f53f80b8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f5d9b640 1 -- 172.21.15.40:0/3183585744 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80b4150 con 0x7f53f80b3500 2024-04-02T16:12:33.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f5d9b640 1 --2- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80b3500 0x7f53f814ae50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097c0 tx=0x7f53ec01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60efcf1f3c5067a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f559a640 1 -- 172.21.15.40:0/3183585744 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0b0070 con 0x7f53f80b3500 2024-04-02T16:12:33.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f559a640 1 -- 172.21.15.40:0/3183585744 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec0ab600 con 0x7f53f80b3500 2024-04-02T16:12:33.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53f559a640 1 -- 172.21.15.40:0/3183585744 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0ab9d0 con 0x7f53f80b3500 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 -- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80b3500 msgr2=0x7f53f814ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 --2- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80b3500 0x7f53f814ae50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0097c0 tx=0x7f53ec01c380 comp rx=0 tx=0).stop 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 -- 172.21.15.40:0/3183585744 shutdown_connections 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 --2- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f80b3500 0x7f53f814ae50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 --2- 172.21.15.40:0/3183585744 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 -- 172.21.15.40:0/3183585744 >> 172.21.15.40:0/3183585744 conn(0x7f53f80734b0 msgr2=0x7f53f80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 -- 172.21.15.40:0/3183585744 shutdown_connections 2024-04-02T16:12:33.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.280+0000 7f53fc992640 1 -- 172.21.15.40:0/3183585744 wait complete. 2024-04-02T16:12:33.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 Processor -- start 2024-04-02T16:12:33.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- start start 2024-04-02T16:12:33.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f8147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f814b390 0x7f53f81e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8154c10 con 0x7f53f80b3500 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8154d90 con 0x7f53f814b390 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f53f8154f10 con 0x7f53f80b4bf0 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40902/0 (socket says 172.21.15.40:40902) 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 -- 172.21.15.40:0/1114492533 learned_addr learned my addr 172.21.15.40:0/1114492533 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f6d9d640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f814b390 0x7f53f81e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f5d9b640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f8147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 -- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 msgr2=0x7f53f8147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f8147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 -- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f814b390 msgr2=0x7f53f81e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f814b390 0x7f53f81e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 -- 172.21.15.40:0/1114492533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8143320 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53f659c640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53e800e960 tx=0x7f53e800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8017070 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e800ca00 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e800cdb0 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f81413a0 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f8143590 con 0x7f53f80b3500 2024-04-02T16:12:33.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.284+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80685f0 con 0x7f53f80b3500 2024-04-02T16:12:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f53e801a070 con 0x7f53f80b3500 2024-04-02T16:12:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53e6ffd640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 0x7f53d806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f53e80871f0 con 0x7f53f80b3500 2024-04-02T16:12:33.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53f5d9b640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 0x7f53d806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53f5d9b640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 0x7f53d806a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad000 tx=0x7f53ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.288+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f53e8057820 con 0x7f53f80b3500 2024-04-02T16:12:33.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.448+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53f80bb0d0 con 0x7f53f80b3500 2024-04-02T16:12:33.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.448+0000 7f53e6ffd640 1 -- 172.21.15.40:0/1114492533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16621 (secure 0 0 0) 0x7f53e8057120 con 0x7f53f80b3500 2024-04-02T16:12:33.451 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:33.451 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:16.692754+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-02T16:12:13.528574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-02T16:12:14.023094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 msgr2=0x7f53d806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 0x7f53d806a840 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad000 tx=0x7f53ec008040 comp rx=0 tx=0).stop 2024-04-02T16:12:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 msgr2=0x7f53f81409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f53e800e960 tx=0x7f53e800ee20 comp rx=0 tx=0).stop 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 shutdown_connections 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53f80b4bf0 0x7f53f8147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f53d8068380 0x7f53d806a840 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53f814b390 0x7f53f81e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 --2- 172.21.15.40:0/1114492533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80b3500 0x7f53f81409b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 >> 172.21.15.40:0/1114492533 conn(0x7f53f80734b0 msgr2=0x7f53f80ba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 shutdown_connections 2024-04-02T16:12:33.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.452+0000 7f53fc992640 1 -- 172.21.15.40:0/1114492533 wait complete. 2024-04-02T16:12:33.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T16:12:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 Processor -- start 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 -- start start 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 0x7f79b0141440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79b00b5860 0x7f79b0141980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b00bb850 con 0x7f79b00bb470 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f79b006d0d0 con 0x7f79b00b5860 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79b4bce640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79b00b6300 con 0x7f79b006e070 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79aed9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79b00b5860 0x7f79b0141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79ae59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f79b00bb470 0x7f79b006d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54204/0 (socket says 172.21.15.40:54204) 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79ae59c640 1 -- 172.21.15.40:0/276142512 learned_addr learned my addr 172.21.15.40:0/276142512 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79add9b640 1 -- 172.21.15.40:0/276142512 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228706550 0 0) 0x7f79b00bb850 con 0x7f79b00bb470 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79add9b640 1 -- 172.21.15.40:0/276142512 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f799c003530 con 0x7f79b00bb470 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79a7fff640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 0x7f79b0141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79aed9d640 1 -- 172.21.15.40:0/276142512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 msgr2=0x7f79b0141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79aed9d640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 0x7f79b0141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79aed9d640 1 -- 172.21.15.40:0/276142512 >> v1:172.21.15.40:6789/0 conn(0x7f79b00bb470 legacy=0x7f79b006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79aed9d640 1 -- 172.21.15.40:0/276142512 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b00b6b90 con 0x7f79b00b5860 2024-04-02T16:12:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.580+0000 7f79a7fff640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 0x7f79b0141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79aed9d640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b5860 0x7f79b0141980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d505b6f38048e7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79add9b640 1 -- 172.21.15.40:0/276142512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b2070 con 0x7f79b00b5860 2024-04-02T16:12:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79add9b640 1 -- 172.21.15.40:0/276142512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a80a59a0 con 0x7f79b00b5860 2024-04-02T16:12:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79add9b640 1 -- 172.21.15.40:0/276142512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80a5d70 con 0x7f79b00b5860 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/276142512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b5860 msgr2=0x7f79b0141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b5860 0x7f79b0141980 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a801c8e0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/276142512 shutdown_connections 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b5860 0x7f79b0141980 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- 172.21.15.40:0/276142512 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b006e070 0x7f79b0141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/276142512 >> 172.21.15.40:0/276142512 conn(0x7f79b002c5d0 msgr2=0x7f79b002c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/276142512 shutdown_connections 2024-04-02T16:12:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/276142512 wait complete. 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 Processor -- start 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- start start 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79b00b5860 0x7f79b00ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b00bb470 0x7f79b00ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0154860 con 0x7f79b006e070 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0073420 con 0x7f79b00b5860 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f79b01549e0 con 0x7f79b00bb470 2024-04-02T16:12:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40914/0 (socket says 172.21.15.40:40914) 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 -- 172.21.15.40:0/4286163708 learned_addr learned my addr 172.21.15.40:0/4286163708 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 -- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b00bb470 msgr2=0x7f79b00ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b00bb470 0x7f79b00ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 -- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79b00b5860 msgr2=0x7f79b00ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79b00b5860 0x7f79b00ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 -- 172.21.15.40:0/4286163708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b01e4200 con 0x7f79b006e070 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79ae59c640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c3c0 tx=0x7f79a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0017070 con 0x7f79b006e070 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0005db0 con 0x7f79b006e070 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a00072f0 con 0x7f79b006e070 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b00afd30 con 0x7f79b006e070 2024-04-02T16:12:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.584+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b0144190 con 0x7f79b006e070 2024-04-02T16:12:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b0068d00 con 0x7f79b006e070 2024-04-02T16:12:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f79a001a070 con 0x7f79b006e070 2024-04-02T16:12:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79a6ffd640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 0x7f799006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f79a0085d20 con 0x7f79b006e070 2024-04-02T16:12:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79a7fff640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 0x7f799006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.588+0000 7f79a7fff640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 0x7f799006a870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79b01422f0 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.592+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f79a0056370 con 0x7f79b006e070 2024-04-02T16:12:33.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.784+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f79b013fdc0 con 0x7f79b006e070 2024-04-02T16:12:33.788 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-04-02T16:12:33.788+0000 7f8eb3a99640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T16:12:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.788+0000 7f79a6ffd640 1 -- 172.21.15.40:0/4286163708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7f79a005c020 con 0x7f79b006e070 2024-04-02T16:12:33.790 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T16:12:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 msgr2=0x7f799006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 0x7f799006a870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79b01422f0 tx=0x7f7998008040 comp rx=0 tx=0).stop 2024-04-02T16:12:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 msgr2=0x7f79b01e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c3c0 tx=0x7f79a000c880 comp rx=0 tx=0).stop 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 shutdown_connections 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79b00bb470 0x7f79b00ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f79900683b0 0x7f799006a870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79b00b5860 0x7f79b00ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 --2- 172.21.15.40:0/4286163708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b006e070 0x7f79b01e3540 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 >> 172.21.15.40:0/4286163708 conn(0x7f79b002c5d0 msgr2=0x7f79b01405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 shutdown_connections 2024-04-02T16:12:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.792+0000 7f79b4bce640 1 -- 172.21.15.40:0/4286163708 wait complete. 2024-04-02T16:12:33.801 INFO:tasks.ceph.mds.a.smithi040.stderr: -2559> 2024-04-02T16:12:19.648+0000 7f5e585c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T16:12:33.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T16:12:33.865 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-04-02T16:12:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 Processor -- start 2024-04-02T16:12:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 -- start start 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6a80b3500 0x7fd6a80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a80b6310 con 0x7fd6a80b4bf0 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a80b55d0 con 0x7fd6a80b8c80 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6af08f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a814b560 con 0x7fd6a80b3500 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6a7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6a80b3500 0x7fd6a80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6ace2b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd6a80b4bf0 0x7fd6a814ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54234/0 (socket says 172.21.15.40:54234) 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6ace2b640 1 -- 172.21.15.40:0/594430635 learned_addr learned my addr 172.21.15.40:0/594430635 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6a7fff640 1 -- 172.21.15.40:0/594430635 >> v1:172.21.15.204:6789/0 conn(0x7fd6a80b8c80 legacy=0x7fd6a81521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6a7fff640 1 -- 172.21.15.40:0/594430635 >> v1:172.21.15.40:6789/0 conn(0x7fd6a80b4bf0 legacy=0x7fd6a814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6a77fe640 1 -- 172.21.15.40:0/594430635 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123033941 0 0) 0x7fd6a80b6310 con 0x7fd6a80b4bf0 2024-04-02T16:12:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.916+0000 7fd6a7fff640 1 -- 172.21.15.40:0/594430635 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a80b3fa0 con 0x7fd6a80b3500 2024-04-02T16:12:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6a7fff640 1 --2- 172.21.15.40:0/594430635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a80b3500 0x7fd6a80b4fd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0097c0 tx=0x7fd69c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdba2eb73dbc8969 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6a77fe640 1 -- 172.21.15.40:0/594430635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0b0070 con 0x7fd6a80b3500 2024-04-02T16:12:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6a77fe640 1 -- 172.21.15.40:0/594430635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c0ab600 con 0x7fd6a80b3500 2024-04-02T16:12:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6a77fe640 1 -- 172.21.15.40:0/594430635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0ab9d0 con 0x7fd6a80b3500 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- 172.21.15.40:0/594430635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a80b3500 msgr2=0x7fd6a80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 --2- 172.21.15.40:0/594430635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a80b3500 0x7fd6a80b4fd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0097c0 tx=0x7fd69c01c340 comp rx=0 tx=0).stop 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- 172.21.15.40:0/594430635 shutdown_connections 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 --2- 172.21.15.40:0/594430635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a80b3500 0x7fd6a80b4fd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- 172.21.15.40:0/594430635 >> 172.21.15.40:0/594430635 conn(0x7fd6a80734b0 msgr2=0x7fd6a80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- 172.21.15.40:0/594430635 shutdown_connections 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- 172.21.15.40:0/594430635 wait complete. 2024-04-02T16:12:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 Processor -- start 2024-04-02T16:12:33.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.920+0000 7fd6af08f640 1 -- start start 2024-04-02T16:12:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a80b4bf0 0x7fd6a814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 0x7fd6a812ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8156740 con 0x7fd6a81485e0 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81568c0 con 0x7fd6a80b4bf0 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a8156a40 con 0x7fd6a80b8c80 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40954/0 (socket says 172.21.15.40:40954) 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 -- 172.21.15.40:0/2036117866 learned_addr learned my addr 172.21.15.40:0/2036117866 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a7fff640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 0x7fd6a812ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ace2b640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a80b4bf0 0x7fd6a814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 -- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 msgr2=0x7fd6a812ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 0x7fd6a812ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 -- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a80b4bf0 msgr2=0x7fd6a814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a80b4bf0 0x7fd6a814ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 -- 172.21.15.40:0/2036117866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a813c230 con 0x7fd6a81485e0 2024-04-02T16:12:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a7fff640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 0x7fd6a812ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ad62c640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b970 tx=0x7fd6a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0019070 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a000cb40 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a813c500 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0014440 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a813cb50 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a80685f0 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd6a001c070 con 0x7fd6a81485e0 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 0x7fd69406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7fd6a0086870 con 0x7fd6a81485e0 2024-04-02T16:12:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ace2b640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 0x7fd69406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.924+0000 7fd6ace2b640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 0x7fd69406a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd698000f70 tx=0x7fd698007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:33.928+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd6a0056f40 con 0x7fd6a81485e0 2024-04-02T16:12:34.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.124+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd6a80b8580 con 0x7fd6a81485e0 2024-04-02T16:12:34.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.796+0000 7fd6a57fa640 1 -- 172.21.15.40:0/2036117866 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7fd6a0056840 con 0x7fd6a81485e0 2024-04-02T16:12:34.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 msgr2=0x7fd69406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 0x7fd69406a840 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd698000f70 tx=0x7fd698007b60 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 msgr2=0x7fd6a812f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000b970 tx=0x7fd6a000be30 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 shutdown_connections 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6a80b8c80 0x7fd6a812ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd694068380 0x7fd69406a840 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6a81485e0 0x7fd6a812f250 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 --2- 172.21.15.40:0/2036117866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a80b4bf0 0x7fd6a814ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 >> 172.21.15.40:0/2036117866 conn(0x7fd6a80734b0 msgr2=0x7fd6a80baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 shutdown_connections 2024-04-02T16:12:34.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.800+0000 7fd6af08f640 1 -- 172.21.15.40:0/2036117866 wait complete. 2024-04-02T16:12:34.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T16:12:34.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 Processor -- start 2024-04-02T16:12:34.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 -- start start 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c0b5b50 con 0x7f8b4c14ae10 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c0bdde0 con 0x7f8b4c0bda00 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b53d15640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4c0b7340 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b512b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b522b2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8b4c14ae10 0x7f8b4c14b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54236/0 (socket says 172.21.15.40:54236) 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b512b0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40966/0 (socket says 172.21.15.40:40966) 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b522b2640 1 -- 172.21.15.40:0/1112938554 learned_addr learned my addr 172.21.15.40:0/1112938554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b512b0640 1 -- 172.21.15.40:0/1112938554 >> v1:172.21.15.204:6789/0 conn(0x7f8b4c0bda00 legacy=0x7f8b4c1415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b512b0640 1 -- 172.21.15.40:0/1112938554 >> v1:172.21.15.40:6789/0 conn(0x7f8b4c14ae10 legacy=0x7f8b4c14b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.956+0000 7f8b512b0640 1 -- 172.21.15.40:0/1112938554 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c0b5df0 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b512b0640 1 --2- 172.21.15.40:0/1112938554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b480097c0 tx=0x7f8b4801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f608b2e39211eec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b50aaf640 1 -- 172.21.15.40:0/1112938554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b480b0070 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b50aaf640 1 -- 172.21.15.40:0/1112938554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b480ab600 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b50aaf640 1 -- 172.21.15.40:0/1112938554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b480ab9d0 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 -- 172.21.15.40:0/1112938554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 msgr2=0x7f8b4c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 --2- 172.21.15.40:0/1112938554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8b480097c0 tx=0x7f8b4801c3a0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 -- 172.21.15.40:0/1112938554 shutdown_connections 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 --2- 172.21.15.40:0/1112938554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c0b52b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 -- 172.21.15.40:0/1112938554 >> 172.21.15.40:0/1112938554 conn(0x7f8b4c073340 msgr2=0x7f8b4c0bd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 -- 172.21.15.40:0/1112938554 shutdown_connections 2024-04-02T16:12:34.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.960+0000 7f8b53d15640 1 -- 172.21.15.40:0/1112938554 wait complete. 2024-04-02T16:12:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 Processor -- start 2024-04-02T16:12:34.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- start start 2024-04-02T16:12:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0bda00 0x7f8b4c142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c14ae10 0x7f8b4c137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c155bb0 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c155a30 con 0x7f8b4c0bda00 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c155d30 con 0x7f8b4c14ae10 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40976/0 (socket says 172.21.15.40:40976) 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 -- 172.21.15.40:0/359131735 learned_addr learned my addr 172.21.15.40:0/359131735 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b512b0640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0bda00 0x7f8b4c142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b522b2640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c14ae10 0x7f8b4c137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 -- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c14ae10 msgr2=0x7f8b4c137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c14ae10 0x7f8b4c137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 -- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0bda00 msgr2=0x7f8b4c142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0bda00 0x7f8b4c142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 -- 172.21.15.40:0/359131735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c13c9e0 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b51ab1640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00c7c0 tx=0x7f8b3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c016020 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c00aa50 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c148240 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00ae40 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c13c700 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c0685f0 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.964+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8b3c019020 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.968+0000 7f8b427fc640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 0x7f8b3006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.968+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f8b3c086a00 con 0x7f8b4c0b4e30 2024-04-02T16:12:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.968+0000 7f8b512b0640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 0x7f8b3006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:34.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.968+0000 7f8b512b0640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 0x7f8b3006a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8b480ad000 tx=0x7f8b48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:34.972+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f8b3c020030 con 0x7f8b4c0b4e30 2024-04-02T16:12:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.136+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8b4c148610 con 0x7f8b4c0b4e30 2024-04-02T16:12:35.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.968+0000 7f8b427fc640 1 -- 172.21.15.40:0/359131735 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f8b3c056ff0 con 0x7f8b4c0b4e30 2024-04-02T16:12:35.970 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 msgr2=0x7f8b3006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 0x7f8b3006a840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8b480ad000 tx=0x7f8b48008040 comp rx=0 tx=0).stop 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 msgr2=0x7f8b4c1e37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00c7c0 tx=0x7f8b3c00cc80 comp rx=0 tx=0).stop 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 shutdown_connections 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c14ae10 0x7f8b4c137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:35.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8b30068380 0x7f8b3006a840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0bda00 0x7f8b4c142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 --2- 172.21.15.40:0/359131735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b4c0b4e30 0x7f8b4c1e37a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 >> 172.21.15.40:0/359131735 conn(0x7f8b4c073340 msgr2=0x7f8b4c141370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 shutdown_connections 2024-04-02T16:12:35.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:35.972+0000 7f8b53d15640 1 -- 172.21.15.40:0/359131735 wait complete. 2024-04-02T16:12:35.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T16:12:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 Processor -- start 2024-04-02T16:12:36.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 -- start start 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8d2406d830 0x7f8d240b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d2414dd80 0x7f8d24150170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d240b4ec0 con 0x7f8d240b5320 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8d240b2c40 con 0x7f8d2414dd80 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d29dc2640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8d240b2560 con 0x7f8d2406d830 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d2414dd80 0x7f8d24150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d2414dd80 0x7f8d24150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50392/0 (socket says 172.21.15.40:50392) 2024-04-02T16:12:36.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 -- 172.21.15.40:0/2770078524 learned_addr learned my addr 172.21.15.40:0/2770078524 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:36.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d237fe640 1 --2- 172.21.15.40:0/2770078524 >> v2:172.21.15.204:3300/0 conn(0x7f8d2406d830 0x7f8d240b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 -- 172.21.15.40:0/2770078524 >> v2:172.21.15.204:3300/0 conn(0x7f8d2406d830 msgr2=0x7f8d240b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:36.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 --2- 172.21.15.40:0/2770078524 >> v2:172.21.15.204:3300/0 conn(0x7f8d2406d830 0x7f8d240b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 -- 172.21.15.40:0/2770078524 >> v1:172.21.15.204:6789/0 conn(0x7f8d240b5320 legacy=0x7f8d24072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:36.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.132+0000 7f8d23fff640 1 -- 172.21.15.40:0/2770078524 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2406dc10 con 0x7f8d2414dd80 2024-04-02T16:12:36.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d23fff640 1 --2- 172.21.15.40:0/2770078524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d24150170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b6740 tx=0x7f8d1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665f13063e3942fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:36.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d227fc640 1 -- 172.21.15.40:0/2770078524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0a5620 con 0x7f8d2414dd80 2024-04-02T16:12:36.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d227fc640 1 -- 172.21.15.40:0/2770078524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0a57c0 con 0x7f8d2414dd80 2024-04-02T16:12:36.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d227fc640 1 -- 172.21.15.40:0/2770078524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0a5b90 con 0x7f8d2414dd80 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- 172.21.15.40:0/2770078524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 msgr2=0x7f8d24150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/2770078524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d24150170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b6740 tx=0x7f8d1c01c960 comp rx=0 tx=0).stop 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- 172.21.15.40:0/2770078524 shutdown_connections 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/2770078524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d24150170 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/2770078524 >> v2:172.21.15.204:3300/0 conn(0x7f8d2406d830 0x7f8d240b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- 172.21.15.40:0/2770078524 >> 172.21.15.40:0/2770078524 conn(0x7f8d240b5890 msgr2=0x7f8d240b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- 172.21.15.40:0/2770078524 shutdown_connections 2024-04-02T16:12:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- 172.21.15.40:0/2770078524 wait complete. 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 Processor -- start 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- start start 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 0x7f8d2414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d240b5320 0x7f8d2414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8d24073480 con 0x7f8d2414dd80 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d240b17d0 con 0x7f8d240b5320 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.136+0000 7f8d29dc2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8d240b2f00 con 0x7f8d2406d830 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50394/0 (socket says 172.21.15.40:50394) 2024-04-02T16:12:36.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 -- 172.21.15.40:0/1687534921 learned_addr learned my addr 172.21.15.40:0/1687534921 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d22ffd640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d240b5320 0x7f8d2414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d237fe640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 0x7f8d2414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 -- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 msgr2=0x7f8d2414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 0x7f8d2414fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 -- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d240b5320 msgr2=0x7f8d2414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d240b5320 0x7f8d2414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 -- 172.21.15.40:0/1687534921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2414d3c0 con 0x7f8d2414dd80 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d237fe640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 0x7f8d2414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d23fff640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c019570 tx=0x7f8d1c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0bc020 con 0x7f8d2414dd80 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0b8670 con 0x7f8d2414dd80 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2414d640 con 0x7f8d2414dd80 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c009510 con 0x7f8d2414dd80 2024-04-02T16:12:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2414c220 con 0x7f8d2414dd80 2024-04-02T16:12:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d240685f0 con 0x7f8d2414dd80 2024-04-02T16:12:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8d1c0bf020 con 0x7f8d2414dd80 2024-04-02T16:12:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 0x7f8d0406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5357+0+0 (secure 0 0 0) 0x7f8d1c121690 con 0x7f8d2414dd80 2024-04-02T16:12:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.140+0000 7f8d237fe640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 0x7f8d0406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.144+0000 7f8d237fe640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 0x7f8d0406a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009840 tx=0x7f8d0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.144+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f8d1c0c6030 con 0x7f8d2414dd80 2024-04-02T16:12:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.328+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d240af2c0 con 0x7f8d2414dd80 2024-04-02T16:12:36.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.980+0000 7f8d1bfff640 1 -- 172.21.15.40:0/1687534921 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f8d2414d3c0 con 0x7f8d2414dd80 2024-04-02T16:12:36.980 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 msgr2=0x7f8d0406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 0x7f8d0406a840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009840 tx=0x7f8d0c008040 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 msgr2=0x7f8d2414abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c019570 tx=0x7f8d1c019700 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 shutdown_connections 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d04068380 0x7f8d0406a840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2414dd80 0x7f8d2414abf0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d240b5320 0x7f8d2414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 --2- 172.21.15.40:0/1687534921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d2406d830 0x7f8d2414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 >> 172.21.15.40:0/1687534921 conn(0x7f8d240b5890 msgr2=0x7f8d240b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 shutdown_connections 2024-04-02T16:12:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:36.984+0000 7f8d29dc2640 1 -- 172.21.15.40:0/1687534921 wait complete. 2024-04-02T16:12:37.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-02T16:12:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 Processor -- start 2024-04-02T16:12:37.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 -- start start 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d140b6dd0 0x7f3d14148c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 0x7f3d141491b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 0x7f3d140b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3d140b5dd0 con 0x7f3d140b6dd0 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3d140b5780 con 0x7f3d140b8210 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1bc50640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d140b5a60 con 0x7f3d140b71b0 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d140b6dd0 0x7f3d14148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3d140b6dd0 0x7f3d14148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50402/0 (socket says 172.21.15.40:50402) 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 -- 172.21.15.40:0/522290027 learned_addr learned my addr 172.21.15.40:0/522290027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d191eb640 1 --2- 172.21.15.40:0/522290027 >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 0x7f3d141491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1a1ed640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 0x7f3d140b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 -- 172.21.15.40:0/522290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 msgr2=0x7f3d140b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 0x7f3d140b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 -- 172.21.15.40:0/522290027 >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 msgr2=0x7f3d141491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 --2- 172.21.15.40:0/522290027 >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 0x7f3d141491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d199ec640 1 -- 172.21.15.40:0/522290027 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d14073440 con 0x7f3d140b6dd0 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d191eb640 1 --2- 172.21.15.40:0/522290027 >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 0x7f3d141491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.136+0000 7f3d1a1ed640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 0x7f3d140b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:37.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d199ec640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b6dd0 0x7f3d14148c70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d140b6700 tx=0x7f3d1001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b369564ac40d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d189ea640 1 -- 172.21.15.40:0/522290027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100b2070 con 0x7f3d140b6dd0 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d189ea640 1 -- 172.21.15.40:0/522290027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100a9070 con 0x7f3d140b6dd0 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d189ea640 1 -- 172.21.15.40:0/522290027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100ad880 con 0x7f3d140b6dd0 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- 172.21.15.40:0/522290027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b6dd0 msgr2=0x7f3d14148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b6dd0 0x7f3d14148c70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d140b6700 tx=0x7f3d1001c490 comp rx=0 tx=0).stop 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- 172.21.15.40:0/522290027 shutdown_connections 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b71b0 0x7f3d140b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/522290027 >> v2:172.21.15.204:3300/0 conn(0x7f3d140b8210 0x7f3d141491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/522290027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b6dd0 0x7f3d14148c70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- 172.21.15.40:0/522290027 >> 172.21.15.40:0/522290027 conn(0x7f3d1406b980 msgr2=0x7f3d1406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- 172.21.15.40:0/522290027 shutdown_connections 2024-04-02T16:12:37.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- 172.21.15.40:0/522290027 wait complete. 2024-04-02T16:12:37.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 Processor -- start 2024-04-02T16:12:37.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- start start 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d140b6dd0 0x7f3d1413f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b8210 0x7f3d141360a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14072420 con 0x7f3d140b71b0 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3d140722a0 con 0x7f3d140b6dd0 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1bc50640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3d140725a0 con 0x7f3d140b8210 2024-04-02T16:12:37.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50404/0 (socket says 172.21.15.40:50404) 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 -- 172.21.15.40:0/2856453268 learned_addr learned my addr 172.21.15.40:0/2856453268 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d199ec640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d140b6dd0 0x7f3d1413f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d1a1ed640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b8210 0x7f3d141360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 -- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b8210 msgr2=0x7f3d141360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b8210 0x7f3d141360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 -- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d140b6dd0 msgr2=0x7f3d1413f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d140b6dd0 0x7f3d1413f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 -- 172.21.15.40:0/2856453268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1413ad50 con 0x7f3d140b71b0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.140+0000 7f3d191eb640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c7c0 tx=0x7f3d0400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04007a30 con 0x7f3d140b71b0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04007bd0 con 0x7f3d140b71b0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1413b030 con 0x7f3d140b71b0 2024-04-02T16:12:37.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040113f0 con 0x7f3d140b71b0 2024-04-02T16:12:37.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1406dda0 con 0x7f3d140b71b0 2024-04-02T16:12:37.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d140685f0 con 0x7f3d140b71b0 2024-04-02T16:12:37.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3d04011590 con 0x7f3d140b71b0 2024-04-02T16:12:37.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3d040867c0 con 0x7f3d140b71b0 2024-04-02T16:12:37.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.144+0000 7f3d199ec640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.148+0000 7f3d199ec640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 0x7f3cf806a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d100af000 tx=0x7f3d100a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.148+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f3d04015020 con 0x7f3d140b71b0 2024-04-02T16:12:37.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.316+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3d140baa70 con 0x7f3d140b71b0 2024-04-02T16:12:37.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.320+0000 7f3d0a7fc640 1 -- 172.21.15.40:0/2856453268 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f3d04057110 con 0x7f3d140b71b0 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 msgr2=0x7f3cf806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 0x7f3cf806a840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d100af000 tx=0x7f3d100a7000 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 msgr2=0x7f3d141e3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c7c0 tx=0x7f3d0400cc80 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 shutdown_connections 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d140b8210 0x7f3d141360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3cf8068380 0x7f3cf806a840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d140b71b0 0x7f3d141e3230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 --2- 172.21.15.40:0/2856453268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d140b6dd0 0x7f3d1413f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.324+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 >> 172.21.15.40:0/2856453268 conn(0x7f3d1406b980 msgr2=0x7f3d14146e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.328+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 shutdown_connections 2024-04-02T16:12:37.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.328+0000 7f3d1bc50640 1 -- 172.21.15.40:0/2856453268 wait complete. 2024-04-02T16:12:37.345 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-02T16:12:37.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.460+0000 7feb18170640 1 Processor -- start 2024-04-02T16:12:37.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.460+0000 7feb18170640 1 -- start start 2024-04-02T16:12:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb18170640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb18170640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 0x7feb1009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb18170640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb100b19e0 con 0x7feb1009dfe0 2024-04-02T16:12:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb18170640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feb100b05c0 con 0x7feb10073230 2024-04-02T16:12:37.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb18170640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb100b08a0 con 0x7feb100b23d0 2024-04-02T16:12:37.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:60920/0 (socket says 172.21.15.40:60920) 2024-04-02T16:12:37.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb1570b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 0x7feb1009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 -- 172.21.15.40:0/2315177466 learned_addr learned my addr 172.21.15.40:0/2315177466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 -- 172.21.15.40:0/2315177466 >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 msgr2=0x7feb1009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 --2- 172.21.15.40:0/2315177466 >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 0x7feb1009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 -- 172.21.15.40:0/2315177466 >> v1:172.21.15.204:6789/0 conn(0x7feb1009dfe0 legacy=0x7feb10148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb15f0c640 1 -- 172.21.15.40:0/2315177466 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb100b0b80 con 0x7feb100b23d0 2024-04-02T16:12:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.464+0000 7feb1570b640 1 --2- 172.21.15.40:0/2315177466 >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 0x7feb1009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:12:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb15f0c640 1 --2- 172.21.15.40:0/2315177466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb08009e50 tx=0x7feb0801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82dd9d873badefcd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb14f0a640 1 -- 172.21.15.40:0/2315177466 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080a5780 con 0x7feb100b23d0 2024-04-02T16:12:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb14f0a640 1 -- 172.21.15.40:0/2315177466 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb080a5920 con 0x7feb100b23d0 2024-04-02T16:12:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb14f0a640 1 -- 172.21.15.40:0/2315177466 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb080a5cf0 con 0x7feb100b23d0 2024-04-02T16:12:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- 172.21.15.40:0/2315177466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 msgr2=0x7feb100b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- 172.21.15.40:0/2315177466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb08009e50 tx=0x7feb0801c9a0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- 172.21.15.40:0/2315177466 shutdown_connections 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- 172.21.15.40:0/2315177466 >> v2:172.21.15.204:3300/0 conn(0x7feb10073230 0x7feb1009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- 172.21.15.40:0/2315177466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb100b23d0 0x7feb100b27b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- 172.21.15.40:0/2315177466 >> 172.21.15.40:0/2315177466 conn(0x7feb100b8fd0 msgr2=0x7feb100b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- 172.21.15.40:0/2315177466 shutdown_connections 2024-04-02T16:12:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- 172.21.15.40:0/2315177466 wait complete. 2024-04-02T16:12:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 Processor -- start 2024-04-02T16:12:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- start start 2024-04-02T16:12:37.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb10073230 0x7feb100aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb100b23d0 0x7feb101e0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb10152a20 con 0x7feb1009dfe0 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb101528a0 con 0x7feb100b23d0 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb18170640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb10152ba0 con 0x7feb10073230 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.468+0000 7feb1570b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50410/0 (socket says 172.21.15.40:50410) 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 -- 172.21.15.40:0/3488372315 learned_addr learned my addr 172.21.15.40:0/3488372315 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1670d640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb100b23d0 0x7feb101e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb15f0c640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb10073230 0x7feb100aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 -- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb10073230 msgr2=0x7feb100aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb10073230 0x7feb100aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 -- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb100b23d0 msgr2=0x7feb101e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb100b23d0 0x7feb101e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 -- 172.21.15.40:0/3488372315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101e3aa0 con 0x7feb1009dfe0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb1570b640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e930 tx=0x7feb0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017070 con 0x7feb1009dfe0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0400ca00 con 0x7feb1009dfe0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400cdd0 con 0x7feb1009dfe0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101e3d40 con 0x7feb1009dfe0 2024-04-02T16:12:37.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101e4710 con 0x7feb1009dfe0 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb100685f0 con 0x7feb1009dfe0 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feb0401a070 con 0x7feb1009dfe0 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 0x7feafc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7feb040865c0 con 0x7feb1009dfe0 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.472+0000 7feb15f0c640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 0x7feafc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:37.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.476+0000 7feb15f0c640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 0x7feafc06a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb080ae000 tx=0x7feb080a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:37.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.476+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7feb04056e70 con 0x7feb1009dfe0 2024-04-02T16:12:37.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.716+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7feb10147d50 con 0x7feb1009dfe0 2024-04-02T16:12:37.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.980+0000 7feb0e7fc640 1 -- 172.21.15.40:0/3488372315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v86) ==== 134+0+0 (secure 0 0 0) 0x7feb04056770 con 0x7feb1009dfe0 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 msgr2=0x7feafc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 0x7feafc06a840 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb080ae000 tx=0x7feb080a7070 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 msgr2=0x7feb1009b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feb0400e930 tx=0x7feb0400edf0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 shutdown_connections 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feafc068380 0x7feafc06a840 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb100b23d0 0x7feb101e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1009dfe0 0x7feb1009b010 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 --2- 172.21.15.40:0/3488372315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb10073230 0x7feb100aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 >> 172.21.15.40:0/3488372315 conn(0x7feb100b8fd0 msgr2=0x7feb100b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 shutdown_connections 2024-04-02T16:12:37.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:37.984+0000 7feb18170640 1 -- 172.21.15.40:0/3488372315 wait complete. 2024-04-02T16:12:38.010 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner: 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-02T16:12:38.011 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 127: 'cephfs-shell -b /tmp/test-cephfsza35oz2z' 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner:Ran 1 test in 42.678s 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner: 2024-04-02T16:12:38.012 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-04-02T16:12:38.013 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi040 with status 127: 'cephfs-shell -b /tmp/test-cephfsza35oz2z' 2024-04-02T16:12:38.014 INFO:tasks.cephfs_test_runner: 2024-04-02T16:12:38.014 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-02T16:12:38.289 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=26eb7be263674166ba9c2b29a12cbc9c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-02T16:12:38.291 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-02T16:12:38.341 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-02T16:12:38.348 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-02T16:12:38.349 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-02T16:12:38.349 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T16:12:38.362 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-02T16:12:38.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:38.362 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-02T16:12:38.409 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T16:12:38.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:38.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T16:12:38.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T16:12:38.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-02T16:12:38.702 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-02T16:12:38.712 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-02T16:12:38.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:38.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 Processor -- start 2024-04-02T16:12:38.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 -- start start 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19c00bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f19c0149590 0x7f19c014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19c00b52e0 con 0x7f19c00b6170 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f19c00b6f00 con 0x7f19c0149590 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c6d31640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19c00b71a0 con 0x7f19c00b4f00 2024-04-02T16:12:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f19c00b6170 0x7f19c0148e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34072/0 (socket says 172.21.15.40:34072) 2024-04-02T16:12:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 -- 172.21.15.40:0/2159440381 learned_addr learned my addr 172.21.15.40:0/2159440381 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c52ce640 1 --2- 172.21.15.40:0/2159440381 >> v2:172.21.15.204:3300/0 conn(0x7f19c0149590 0x7f19c014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c4acd640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19c00bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147965249 0 0) 0x7f19b4001120 con 0x7f19c00b6170 2024-04-02T16:12:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19a8003530 con 0x7f19c00b6170 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1505889256 0 0) 0x7f19a8003530 con 0x7f19c00b6170 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 msgr2=0x7f19c00bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19c00bb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 >> v2:172.21.15.204:3300/0 conn(0x7f19c0149590 msgr2=0x7f19c014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 --2- 172.21.15.40:0/2159440381 >> v2:172.21.15.204:3300/0 conn(0x7f19c0149590 0x7f19c014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c00b7480 con 0x7f19c00b6170 2024-04-02T16:12:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f19b4001850 con 0x7f19c00b6170 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 >> v1:172.21.15.40:6789/0 conn(0x7f19c00b6170 legacy=0x7f19c0148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19a80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19a8004d50 0x7f19a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00b6f00 con 0x7f19a8003f40 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00b71a0 con 0x7f19a8004d50 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19a800bab0 con 0x7f19c00b4f00 2024-04-02T16:12:38.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c4acd640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19a80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19c52ce640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19a8004d50 0x7f19a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 -- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 msgr2=0x7f19a80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c00b4f00 0x7f19a80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 -- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19a8004d50 msgr2=0x7f19a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19a8004d50 0x7f19a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.844+0000 7f19bffff640 1 -- 172.21.15.40:0/2159440381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a8003530 con 0x7f19a8003f40 2024-04-02T16:12:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19bffff640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19a8004810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c4c0 tx=0x7f19b401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca63048aabf58f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b400c0e0 con 0x7f19a8003f40 2024-04-02T16:12:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b40a9020 con 0x7f19a8003f40 2024-04-02T16:12:38.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19bf7fe640 1 -- 172.21.15.40:0/2159440381 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4005ae0 con 0x7f19a8003f40 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- 172.21.15.40:0/2159440381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 msgr2=0x7f19a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 --2- 172.21.15.40:0/2159440381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19a8004810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c4c0 tx=0x7f19b401cc70 comp rx=0 tx=0).stop 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c52ce640 1 -- 172.21.15.40:0/2159440381 reap_dead start 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- 172.21.15.40:0/2159440381 shutdown_connections 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- 172.21.15.40:0/2159440381 >> 172.21.15.40:0/2159440381 conn(0x7f19c002c5d0 msgr2=0x7f19c002c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- 172.21.15.40:0/2159440381 shutdown_connections 2024-04-02T16:12:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- 172.21.15.40:0/2159440381 wait complete. 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 Processor -- start 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- start start 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c014c820 0x7f19c01e1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c01e1cd0 0x7f19c014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19c0150590 con 0x7f19a8003f40 2024-04-02T16:12:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19c00b59e0 con 0x7f19c01e1cd0 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c6d31640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19c0150710 con 0x7f19c014c820 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.848+0000 7f19c4acd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50426/0 (socket says 172.21.15.40:50426) 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 -- 172.21.15.40:0/719964652 learned_addr learned my addr 172.21.15.40:0/719964652 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bffff640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c014c820 0x7f19c01e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c52ce640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c01e1cd0 0x7f19c014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 -- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c014c820 msgr2=0x7f19c01e1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c014c820 0x7f19c01e1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 -- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c01e1cd0 msgr2=0x7f19c014d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c01e1cd0 0x7f19c014d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 -- 172.21.15.40:0/719964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c01e2180 con 0x7f19a8003f40 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c4acd640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f19b0011590 tx=0x7f19b0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0015930 con 0x7f19a8003f40 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b0015ad0 con 0x7f19a8003f40 2024-04-02T16:12:38.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c014fdc0 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b000f460 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c014f440 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c0068d00 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f19b000f600 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 0x7f19ac06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f19b0086e20 con 0x7f19a8003f40 2024-04-02T16:12:38.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.852+0000 7f19bffff640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 0x7f19ac06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.856+0000 7f19bffff640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 0x7f19ac06a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19b40aa000 tx=0x7f19b4001c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:38.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:38.856+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f19c01e2180 con 0x7f19a8003f40 2024-04-02T16:12:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.016+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19c014fc10 con 0x7f19ac0683b0 2024-04-02T16:12:39.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19bd7fa640 1 -- 172.21.15.40:0/719964652 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f19c014fc10 con 0x7f19ac0683b0 2024-04-02T16:12:39.021 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:39.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 msgr2=0x7f19ac06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 0x7f19ac06a870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19b40aa000 tx=0x7f19b4001c90 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 msgr2=0x7f19c014e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f19b0011590 tx=0x7f19b0011a50 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 shutdown_connections 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19c014c820 0x7f19c01e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f19ac0683b0 0x7f19ac06a870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19c01e1cd0 0x7f19c014d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 --2- 172.21.15.40:0/719964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19a8003f40 0x7f19c014e100 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 >> 172.21.15.40:0/719964652 conn(0x7f19c002c5d0 msgr2=0x7f19c00b8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.020+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 shutdown_connections 2024-04-02T16:12:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.024+0000 7f19c6d31640 1 -- 172.21.15.40:0/719964652 wait complete. 2024-04-02T16:12:39.040 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-02T16:12:37.871013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224148,"kb_used_data":3828,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750572,"statfs":{"total":773094113280,"available":772864585728,"internally_reserved":0,"allocated":3919872,"data_stored":2225019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-53,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002498"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-02T16:12:34.041456+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:34.041456+0000","last_peered":"2024-04-02T16:12:34.041456+0000","last_clean":"2024-04-02T16:12:34.041456+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:34.041456+0000","last_undegraded":"2024-04-02T16:12:34.041456+0000","last_fullsized":"2024-04-02T16:12:34.041456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":140208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:39.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:39.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.164+0000 7f94bbfd9640 1 Processor -- start 2024-04-02T16:12:39.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.164+0000 7f94bbfd9640 1 -- start start 2024-04-02T16:12:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94bbfd9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94b4073230 0x7f94b40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94bbfd9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b4149280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94bbfd9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94b40b19e0 con 0x7f94b40b23d0 2024-04-02T16:12:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94bbfd9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f94b41497c0 con 0x7f94b4073230 2024-04-02T16:12:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94bbfd9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f94b40b05c0 con 0x7f94b4148e50 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9d75640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94b40b23d0 0x7f94b409daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34084/0 (socket says 172.21.15.40:34084) 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9d75640 1 -- 172.21.15.40:0/2969317833 learned_addr learned my addr 172.21.15.40:0/2969317833 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 --2- 172.21.15.40:0/2969317833 >> v2:172.21.15.40:3300/0 conn(0x7f94b4073230 0x7f94b40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b8d73640 1 -- 172.21.15.40:0/2969317833 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2945730345 0 0) 0x7f94b40b19e0 con 0x7f94b40b23d0 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b8d73640 1 -- 172.21.15.40:0/2969317833 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f949c003530 con 0x7f94b40b23d0 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94ba576640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b4149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 -- 172.21.15.40:0/2969317833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 msgr2=0x7f94b4149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b4149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 -- 172.21.15.40:0/2969317833 >> v1:172.21.15.40:6789/0 conn(0x7f94b40b23d0 legacy=0x7f94b409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 -- 172.21.15.40:0/2969317833 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b40b08a0 con 0x7f94b4073230 2024-04-02T16:12:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b9574640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40b27b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94a80097c0 tx=0x7f94a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b4e43d591888d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b8d73640 1 -- 172.21.15.40:0/2969317833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ab430 con 0x7f94b4073230 2024-04-02T16:12:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b8d73640 1 -- 172.21.15.40:0/2969317833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a80b0070 con 0x7f94b4073230 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.168+0000 7f94b8d73640 1 -- 172.21.15.40:0/2969317833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80ab7f0 con 0x7f94b4073230 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/2969317833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 msgr2=0x7f94b40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40b27b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94a80097c0 tx=0x7f94a801c410 comp rx=0 tx=0).stop 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/2969317833 shutdown_connections 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b4149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/2969317833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40b27b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/2969317833 >> 172.21.15.40:0/2969317833 conn(0x7f94b40b8fd0 msgr2=0x7f94b40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/2969317833 shutdown_connections 2024-04-02T16:12:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/2969317833 wait complete. 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 Processor -- start 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- start start 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 0x7f94b409b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b41e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4152810 con 0x7f94b4073230 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4152690 con 0x7f94b40b23d0 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f94b4152990 con 0x7f94b4148e50 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50442/0 (socket says 172.21.15.40:50442) 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 -- 172.21.15.40:0/842269346 learned_addr learned my addr 172.21.15.40:0/842269346 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94ba576640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b41e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9574640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 0x7f94b409b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 -- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 msgr2=0x7f94b41e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b41e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 -- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 msgr2=0x7f94b409b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 0x7f94b409b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 -- 172.21.15.40:0/842269346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b409ab10 con 0x7f94b4073230 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9574640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 0x7f94b409b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b9d75640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c450 tx=0x7f94a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x7f94b4073230 2024-04-02T16:12:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a4005cf0 con 0x7f94b4073230 2024-04-02T16:12:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4007290 con 0x7f94b4073230 2024-04-02T16:12:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41479a0 con 0x7f94b4073230 2024-04-02T16:12:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.172+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b409bd50 con 0x7f94b4073230 2024-04-02T16:12:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b409bfe0 con 0x7f94b4073230 2024-04-02T16:12:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f94a401a070 con 0x7f94b4073230 2024-04-02T16:12:39.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94b27fc640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 0x7f94a006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f94a4085950 con 0x7f94b4073230 2024-04-02T16:12:39.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94b9574640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 0x7f94a006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.176+0000 7f94b9574640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 0x7f94a006a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94a80af000 tx=0x7f94a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.180+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f94a4055c50 con 0x7f94b4073230 2024-04-02T16:12:39.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.340+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94b41e03d0 con 0x7f94a0068380 2024-04-02T16:12:39.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.340+0000 7f94b27fc640 1 -- 172.21.15.40:0/842269346 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f94a80b0040 con 0x7f94a0068380 2024-04-02T16:12:39.342 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:39.342 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 msgr2=0x7f94a006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 0x7f94a006a840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94a80af000 tx=0x7f94a8008040 comp rx=0 tx=0).stop 2024-04-02T16:12:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 msgr2=0x7f94b40af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c450 tx=0x7f94a400c910 comp rx=0 tx=0).stop 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 shutdown_connections 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94b4148e50 0x7f94b41e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f94a0068380 0x7f94a006a840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b40b23d0 0x7f94b409b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 --2- 172.21.15.40:0/842269346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b4073230 0x7f94b40af4f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 >> 172.21.15.40:0/842269346 conn(0x7f94b40b8fd0 msgr2=0x7f94b406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 shutdown_connections 2024-04-02T16:12:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.344+0000 7f94bbfd9640 1 -- 172.21.15.40:0/842269346 wait complete. 2024-04-02T16:12:39.361 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-02T16:12:37.871013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224148,"kb_used_data":3828,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750572,"statfs":{"total":773094113280,"available":772864585728,"internally_reserved":0,"allocated":3919872,"data_stored":2225019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-53,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002498"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-02T16:12:34.041456+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:34.041456+0000","last_peered":"2024-04-02T16:12:34.041456+0000","last_clean":"2024-04-02T16:12:34.041456+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:34.041456+0000","last_undegraded":"2024-04-02T16:12:34.041456+0000","last_fullsized":"2024-04-02T16:12:34.041456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":140208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:39.362 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-02T16:12:39.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 Processor -- start 2024-04-02T16:12:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 -- start start 2024-04-02T16:12:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1800071bb0 0x7f1800071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18000708c0 0x7f180014c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1800070d50 con 0x7f18000b3b70 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f180014bd70 con 0x7f18000708c0 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f1807b06640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f180014c050 con 0x7f1800071bb0 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18050a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f18000708c0 0x7f180014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18060a3640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f18000b3b70 0x7f180014f1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34086/0 (socket says 172.21.15.40:34086) 2024-04-02T16:12:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18060a3640 1 -- 172.21.15.40:0/3623625480 learned_addr learned my addr 172.21.15.40:0/3623625480 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18058a2640 1 --2- 172.21.15.40:0/3623625480 >> v2:172.21.15.204:3300/0 conn(0x7f1800071bb0 0x7f1800071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18048a0640 1 -- 172.21.15.40:0/3623625480 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827967375 0 0) 0x7f1800070d50 con 0x7f18000b3b70 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18048a0640 1 -- 172.21.15.40:0/3623625480 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17ec003530 con 0x7f18000b3b70 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18050a1640 1 -- 172.21.15.40:0/3623625480 >> v2:172.21.15.204:3300/0 conn(0x7f1800071bb0 msgr2=0x7f1800071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18050a1640 1 --2- 172.21.15.40:0/3623625480 >> v2:172.21.15.204:3300/0 conn(0x7f1800071bb0 0x7f1800071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18050a1640 1 -- 172.21.15.40:0/3623625480 >> v1:172.21.15.40:6789/0 conn(0x7f18000b3b70 legacy=0x7f180014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.480+0000 7f18050a1640 1 -- 172.21.15.40:0/3623625480 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18000724d0 con 0x7f18000708c0 2024-04-02T16:12:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f18050a1640 1 --2- 172.21.15.40:0/3623625480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18000708c0 0x7f180014c570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f18000b0bf0 tx=0x7f17f001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8551149886fc51e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f18048a0640 1 -- 172.21.15.40:0/3623625480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00b0070 con 0x7f18000708c0 2024-04-02T16:12:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f18048a0640 1 -- 172.21.15.40:0/3623625480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f00ab600 con 0x7f18000708c0 2024-04-02T16:12:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f18048a0640 1 -- 172.21.15.40:0/3623625480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00ab9d0 con 0x7f18000708c0 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- 172.21.15.40:0/3623625480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18000708c0 msgr2=0x7f180014c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 --2- 172.21.15.40:0/3623625480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18000708c0 0x7f180014c570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f18000b0bf0 tx=0x7f17f001c3f0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- 172.21.15.40:0/3623625480 shutdown_connections 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 --2- 172.21.15.40:0/3623625480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18000708c0 0x7f180014c570 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 --2- 172.21.15.40:0/3623625480 >> v2:172.21.15.204:3300/0 conn(0x7f1800071bb0 0x7f1800071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- 172.21.15.40:0/3623625480 >> 172.21.15.40:0/3623625480 conn(0x7f18000b7d10 msgr2=0x7f18000b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- 172.21.15.40:0/3623625480 shutdown_connections 2024-04-02T16:12:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- 172.21.15.40:0/3623625480 wait complete. 2024-04-02T16:12:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 Processor -- start 2024-04-02T16:12:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.484+0000 7f1807b06640 1 -- start start 2024-04-02T16:12:39.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1800071bb0 0x7f180006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18000b3b70 0x7f180009b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f18001596a0 con 0x7f18001e4920 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1800159520 con 0x7f1800071bb0 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1800159820 con 0x7f18000b3b70 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50458/0 (socket says 172.21.15.40:50458) 2024-04-02T16:12:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 -- 172.21.15.40:0/2758849382 learned_addr learned my addr 172.21.15.40:0/2758849382 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18050a1640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18000b3b70 0x7f180009b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18058a2640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1800071bb0 0x7f180006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 -- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18000b3b70 msgr2=0x7f180009b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18000b3b70 0x7f180009b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 -- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1800071bb0 msgr2=0x7f180006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1800071bb0 0x7f180006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 -- 172.21.15.40:0/2758849382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1800147df0 con 0x7f18001e4920 2024-04-02T16:12:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18060a3640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f17f800caa0 tx=0x7f17f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8017070 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f800ac00 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180009b720 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8012400 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1800148c50 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1800068d00 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f17f801a070 con 0x7f18001e4920 2024-04-02T16:12:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 0x7f17e406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f17f8086770 con 0x7f18001e4920 2024-04-02T16:12:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.488+0000 7f18058a2640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 0x7f17e406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.492+0000 7f18058a2640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 0x7f17e406a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b9d0 tx=0x7f17fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.492+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f17f8021020 con 0x7f18001e4920 2024-04-02T16:12:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.644+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18000b0180 con 0x7f17e4068380 2024-04-02T16:12:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f17f67fc640 1 -- 172.21.15.40:0/2758849382 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f18000b0180 con 0x7f17e4068380 2024-04-02T16:12:39.649 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 msgr2=0x7f17e406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 0x7f17e406a840 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b9d0 tx=0x7f17fc008040 comp rx=0 tx=0).stop 2024-04-02T16:12:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 msgr2=0x7f1800145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f17f800caa0 tx=0x7f17f800cf60 comp rx=0 tx=0).stop 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 shutdown_connections 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18000b3b70 0x7f180009b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f17e4068380 0x7f17e406a840 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18001e4920 0x7f1800145800 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 --2- 172.21.15.40:0/2758849382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1800071bb0 0x7f180006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 >> 172.21.15.40:0/2758849382 conn(0x7f18000b7d10 msgr2=0x7f18000b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 shutdown_connections 2024-04-02T16:12:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.648+0000 7f1807b06640 1 -- 172.21.15.40:0/2758849382 wait complete. 2024-04-02T16:12:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-02T16:12:37.871013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224148,"kb_used_data":3828,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750572,"statfs":{"total":773094113280,"available":772864585728,"internally_reserved":0,"allocated":3919872,"data_stored":2225019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-53,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002498"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-02T16:12:34.041456+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:34.041456+0000","last_peered":"2024-04-02T16:12:34.041456+0000","last_clean":"2024-04-02T16:12:34.041456+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:34.041456+0000","last_undegraded":"2024-04-02T16:12:34.041456+0000","last_fullsized":"2024-04-02T16:12:34.041456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":360448,"data_stored":140208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:39.667 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T16:12:39.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.784+0000 7f37dffef640 1 Processor -- start 2024-04-02T16:12:39.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.784+0000 7f37dffef640 1 -- start start 2024-04-02T16:12:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d80b23d0 0x7f37d80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d80b19e0 con 0x7f37d809dfe0 2024-04-02T16:12:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f37d80b05c0 con 0x7f37d80b23d0 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37d80b08a0 con 0x7f37d8073230 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d80b23d0 0x7f37d80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37d80b23d0 0x7f37d80b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50470/0 (socket says 172.21.15.40:50470) 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 -- 172.21.15.40:0/1245697049 learned_addr learned my addr 172.21.15.40:0/1245697049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dd58a640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 -- 172.21.15.40:0/1245697049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 msgr2=0x7f37d809daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 -- 172.21.15.40:0/1245697049 >> v1:172.21.15.204:6789/0 conn(0x7f37d809dfe0 legacy=0x7f37d8148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 -- 172.21.15.40:0/1245697049 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d80b0b80 con 0x7f37d80b23d0 2024-04-02T16:12:39.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37ddd8b640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80b27b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009870 tx=0x7f37d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d5042578b2bf7d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dcd89640 1 -- 172.21.15.40:0/1245697049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d40ab5f0 con 0x7f37d80b23d0 2024-04-02T16:12:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dcd89640 1 -- 172.21.15.40:0/1245697049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d40b0070 con 0x7f37d80b23d0 2024-04-02T16:12:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dcd89640 1 -- 172.21.15.40:0/1245697049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d40ab9b0 con 0x7f37d80b23d0 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- 172.21.15.40:0/1245697049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 msgr2=0x7f37d80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80b27b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009870 tx=0x7f37d401c4d0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- 172.21.15.40:0/1245697049 shutdown_connections 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 --2- 172.21.15.40:0/1245697049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80b27b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- 172.21.15.40:0/1245697049 >> 172.21.15.40:0/1245697049 conn(0x7f37d80b8fd0 msgr2=0x7f37d80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- 172.21.15.40:0/1245697049 shutdown_connections 2024-04-02T16:12:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.788+0000 7f37dffef640 1 -- 172.21.15.40:0/1245697049 wait complete. 2024-04-02T16:12:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 Processor -- start 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- start start 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d809dfe0 0x7f37d80aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8152a20 con 0x7f37d80b23d0 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37d81528a0 con 0x7f37d809dfe0 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37d8152ba0 con 0x7f37d8073230 2024-04-02T16:12:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50474/0 (socket says 172.21.15.40:50474) 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 -- 172.21.15.40:0/1670333914 learned_addr learned my addr 172.21.15.40:0/1670333914 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37ddd8b640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 -- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 msgr2=0x7f37d809acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dd58a640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d809dfe0 0x7f37d80aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 -- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d809dfe0 msgr2=0x7f37d80aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d809dfe0 0x7f37d80aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 -- 172.21.15.40:0/1670333914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d81e12b0 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37de58c640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c770 tx=0x7f37d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0003b80 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d0003d20 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d81e0c60 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000aa90 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d80ae070 con 0x7f37d80b23d0 2024-04-02T16:12:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.792+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d80685f0 con 0x7f37d80b23d0 2024-04-02T16:12:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.796+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f37d000ac30 con 0x7f37d80b23d0 2024-04-02T16:12:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.796+0000 7f37ce7fc640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 0x7f37bc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.796+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f37d0086fb0 con 0x7f37d80b23d0 2024-04-02T16:12:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.796+0000 7f37ddd8b640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 0x7f37bc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.796+0000 7f37ddd8b640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 0x7f37bc06a840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f37d40af000 tx=0x7f37d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.800+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f37d00579e0 con 0x7f37d80b23d0 2024-04-02T16:12:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.960+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37d80b1690 con 0x7f37d80b23d0 2024-04-02T16:12:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37ce7fc640 1 -- 172.21.15.40:0/1670333914 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+12972 (secure 0 0 0) 0x7f37d00572e0 con 0x7f37d80b23d0 2024-04-02T16:12:39.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:39.965 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"fsid":"f7c926e5-333e-4244-9af2-414c6ebf5077","created":"2024-04-02T16:11:07.166707+0000","modified":"2024-04-02T16:12:36.971155+0000","last_up_change":"2024-04-02T16:11:19.018742+0000","last_in_change":"2024-04-02T16:11:09.416797+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T16:11:19.895393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1ec8a9b0-daac-4c76-9cd5-56e61b58c5ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6801","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6803","nonce":519999405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6807","nonce":519999405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6815","nonce":519999405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":519999405},{"type":"v1","addr":"172.21.15.40:6812","nonce":519999405}]},"public_addr":"172.21.15.40:6803/519999405","cluster_addr":"172.21.15.40:6807/519999405","heartbeat_back_addr":"172.21.15.40:6815/519999405","heartbeat_front_addr":"172.21.15.40:6812/519999405","state":["exists","up"]},{"osd":1,"uuid":"2f24db8f-47da-4934-bd1d-8841a29c88ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6817","nonce":422638552}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6819","nonce":422638552}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6823","nonce":422638552}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":422638552},{"type":"v1","addr":"172.21.15.40:6821","nonce":422638552}]},"public_addr":"172.21.15.40:6817/422638552","cluster_addr":"172.21.15.40:6819/422638552","heartbeat_back_addr":"172.21.15.40:6823/422638552","heartbeat_front_addr":"172.21.15.40:6821/422638552","state":["exists","up"]},{"osd":2,"uuid":"7acd5c7b-b1a3-443d-ab00-a6c1ef72ffdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6825","nonce":658893798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6827","nonce":658893798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6831","nonce":658893798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":658893798},{"type":"v1","addr":"172.21.15.40:6829","nonce":658893798}]},"public_addr":"172.21.15.40:6825/658893798","cluster_addr":"172.21.15.40:6827/658893798","heartbeat_back_addr":"172.21.15.40:6831/658893798","heartbeat_front_addr":"172.21.15.40:6829/658893798","state":["exists","up"]},{"osd":3,"uuid":"e06f6d74-bb47-480c-9d67-0b6a76e482b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6802","nonce":1014552006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6806","nonce":1014552006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6813","nonce":1014552006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1014552006},{"type":"v1","addr":"172.21.15.40:6809","nonce":1014552006}]},"public_addr":"172.21.15.40:6802/1014552006","cluster_addr":"172.21.15.40:6806/1014552006","heartbeat_back_addr":"172.21.15.40:6813/1014552006","heartbeat_front_addr":"172.21.15.40:6809/1014552006","state":["exists","up"]},{"osd":4,"uuid":"3e262364-3787-4159-a389-beb019a2305d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6806","nonce":3004039088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6814","nonce":3004039088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6827","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6830","nonce":3004039088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3004039088},{"type":"v1","addr":"172.21.15.204:6822","nonce":3004039088}]},"public_addr":"172.21.15.204:6806/3004039088","cluster_addr":"172.21.15.204:6814/3004039088","heartbeat_back_addr":"172.21.15.204:6830/3004039088","heartbeat_front_addr":"172.21.15.204:6822/3004039088","state":["exists","up"]},{"osd":5,"uuid":"8af58e19-9afb-40f2-a32a-9b94817636d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6801","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6804","nonce":246318220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6811","nonce":246318220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6823","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6826","nonce":246318220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6815","nonce":246318220},{"type":"v1","addr":"172.21.15.204:6819","nonce":246318220}]},"public_addr":"172.21.15.204:6804/246318220","cluster_addr":"172.21.15.204:6811/246318220","heartbeat_back_addr":"172.21.15.204:6826/246318220","heartbeat_front_addr":"172.21.15.204:6819/246318220","state":["exists","up"]},{"osd":6,"uuid":"00d9c46d-e5b7-4097-85fe-8434592d792b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6803","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6807","nonce":1944425714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6816","nonce":1944425714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6831","nonce":1944425714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6821","nonce":1944425714},{"type":"v1","addr":"172.21.15.204:6825","nonce":1944425714}]},"public_addr":"172.21.15.204:6807/1944425714","cluster_addr":"172.21.15.204:6816/1944425714","heartbeat_back_addr":"172.21.15.204:6831/1944425714","heartbeat_front_addr":"172.21.15.204:6825/1944425714","state":["exists","up"]},{"osd":7,"uuid":"242ea867-d592-4b2b-a673-bfd5739b9791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6805","nonce":176733231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6809","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6813","nonce":176733231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6829","nonce":176733231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6817","nonce":176733231},{"type":"v1","addr":"172.21.15.204:6820","nonce":176733231}]},"public_addr":"172.21.15.204:6805/176733231","cluster_addr":"172.21.15.204:6813/176733231","heartbeat_back_addr":"172.21.15.204:6829/176733231","heartbeat_front_addr":"172.21.15.204:6820/176733231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.035309+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.960820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.073973+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.900767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.937355+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.938646+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:14.984788+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T16:11:15.000770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/2966573419":"2024-04-03T16:12:33.788951+0000","172.21.15.40:6832/2966573419":"2024-04-03T16:12:33.788951+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 msgr2=0x7f37bc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 0x7f37bc06a840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f37d40af000 tx=0x7f37d4008040 comp rx=0 tx=0).stop 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 msgr2=0x7f37d80af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c770 tx=0x7f37d000cc30 comp rx=0 tx=0).stop 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 shutdown_connections 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f37bc068380 0x7f37bc06a840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37d80b23d0 0x7f37d80af3f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d809dfe0 0x7f37d80aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 --2- 172.21.15.40:0/1670333914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d8073230 0x7f37d809acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 >> 172.21.15.40:0/1670333914 conn(0x7f37d80b8fd0 msgr2=0x7f37d80b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 shutdown_connections 2024-04-02T16:12:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:39.964+0000 7f37dffef640 1 -- 172.21.15.40:0/1670333914 wait complete. 2024-04-02T16:12:40.985 INFO:tasks.ceph:Scrubbing osd.0 2024-04-02T16:12:40.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 Processor -- start 2024-04-02T16:12:41.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 -- start start 2024-04-02T16:12:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 0x7f07280b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07280b5320 0x7f0728072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280730a0 0x7f07280b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f07280b2c40 con 0x7f07280b5320 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f07280b2560 con 0x7f072806d830 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f072d3a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f07280b2f00 con 0x7f07280730a0 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07280b5320 0x7f0728072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07280b5320 0x7f0728072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50486/0 (socket says 172.21.15.40:50486) 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 -- 172.21.15.40:0/2067367724 learned_addr learned my addr 172.21.15.40:0/2067367724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f0726ffd640 1 --2- 172.21.15.40:0/2067367724 >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 0x7f07280b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 -- 172.21.15.40:0/2067367724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280730a0 msgr2=0x7f07280b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 --2- 172.21.15.40:0/2067367724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280730a0 0x7f07280b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 -- 172.21.15.40:0/2067367724 >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 msgr2=0x7f07280b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 --2- 172.21.15.40:0/2067367724 >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 0x7f07280b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f07267fc640 1 -- 172.21.15.40:0/2067367724 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07280b4ec0 con 0x7f07280b5320 2024-04-02T16:12:41.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.124+0000 7f0726ffd640 1 --2- 172.21.15.40:0/2067367724 >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 0x7f07280b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f07267fc640 1 --2- 172.21.15.40:0/2067367724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280b5320 0x7f0728072b60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07280b6740 tx=0x7f071001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92332a89719109c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f0725ffb640 1 -- 172.21.15.40:0/2067367724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100b0070 con 0x7f07280b5320 2024-04-02T16:12:41.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f0725ffb640 1 -- 172.21.15.40:0/2067367724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07100ab600 con 0x7f07280b5320 2024-04-02T16:12:41.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f0725ffb640 1 -- 172.21.15.40:0/2067367724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100ab9d0 con 0x7f07280b5320 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- 172.21.15.40:0/2067367724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280b5320 msgr2=0x7f0728072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 --2- 172.21.15.40:0/2067367724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280b5320 0x7f0728072b60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07280b6740 tx=0x7f071001c260 comp rx=0 tx=0).stop 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- 172.21.15.40:0/2067367724 shutdown_connections 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 --2- 172.21.15.40:0/2067367724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280730a0 0x7f07280b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 --2- 172.21.15.40:0/2067367724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280b5320 0x7f0728072b60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 --2- 172.21.15.40:0/2067367724 >> v2:172.21.15.204:3300/0 conn(0x7f072806d830 0x7f07280b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- 172.21.15.40:0/2067367724 >> 172.21.15.40:0/2067367724 conn(0x7f07280b5890 msgr2=0x7f07280b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- 172.21.15.40:0/2067367724 shutdown_connections 2024-04-02T16:12:41.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- 172.21.15.40:0/2067367724 wait complete. 2024-04-02T16:12:41.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 Processor -- start 2024-04-02T16:12:41.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.128+0000 7f072d3a0640 1 -- start start 2024-04-02T16:12:41.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f072806d830 0x7f07281ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 0x7f07281478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280b5320 0x7f07281465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0728145470 con 0x7f07280730a0 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f07280b17d0 con 0x7f072806d830 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f07281455f0 con 0x7f07280b5320 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 0x7f07281478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 0x7f07281478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50502/0 (socket says 172.21.15.40:50502) 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 -- 172.21.15.40:0/1979585991 learned_addr learned my addr 172.21.15.40:0/1979585991 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07277fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280b5320 0x7f07281465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f0726ffd640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f072806d830 0x7f07281ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 -- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280b5320 msgr2=0x7f07281465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280b5320 0x7f07281465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 -- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f072806d830 msgr2=0x7f07281ddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f072806d830 0x7f07281ddea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0728148940 con 0x7f07280730a0 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07277fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f07280b5320 0x7f07281465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07267fc640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 0x7f07281478f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f07100ae040 tx=0x7f0710019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100ba020 con 0x7f07280730a0 2024-04-02T16:12:41.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07100aa7d0 con 0x7f07280730a0 2024-04-02T16:12:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0728148c20 con 0x7f07280730a0 2024-04-02T16:12:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100b0040 con 0x7f07280730a0 2024-04-02T16:12:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0728149670 con 0x7f07280730a0 2024-04-02T16:12:41.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07280685f0 con 0x7f07280730a0 2024-04-02T16:12:41.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f07100bd020 con 0x7f07280730a0 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0708068380 0x7f070806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f0710120ff0 con 0x7f07280730a0 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f070806e240 0x7f0708070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0708070d10 con 0x7f070806e240 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0710121480 con 0x7f07280730a0 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.132+0000 7f07277fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f070806e240 0x7f0708070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.136+0000 7f0726ffd640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0708068380 0x7f070806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.136+0000 7f07277fe640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f070806e240 0x7f0708070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.136+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== osd.0 v2:172.21.15.40:6801/519999405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0708070d10 con 0x7f070806e240 2024-04-02T16:12:41.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.136+0000 7f0726ffd640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0708068380 0x7f070806a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0718006fd0 tx=0x7f0718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 --> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f07280b73a0 con 0x7f070806e240 2024-04-02T16:12:41.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f071f7fe640 1 -- 172.21.15.40:0/1979585991 <== osd.0 v2:172.21.15.40:6801/519999405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f07280b73a0 con 0x7f070806e240 2024-04-02T16:12:41.150 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:41.151 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:41.151 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:41.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f070806e240 msgr2=0x7f0708070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:6801/519999405,v1:172.21.15.40:6803/519999405] conn(0x7f070806e240 0x7f0708070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0708068380 msgr2=0x7f070806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f0708068380 0x7f070806a840 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0718006fd0 tx=0x7f0718008040 comp rx=0 tx=0).stop 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 msgr2=0x7f07281478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 --2- 172.21.15.40:0/1979585991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07280730a0 0x7f07281478f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f07100ae040 tx=0x7f0710019000 comp rx=0 tx=0).stop 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f07277fe640 1 -- 172.21.15.40:0/1979585991 reap_dead start 2024-04-02T16:12:41.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 shutdown_connections 2024-04-02T16:12:41.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 >> 172.21.15.40:0/1979585991 conn(0x7f07280b5890 msgr2=0x7f07280b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 shutdown_connections 2024-04-02T16:12:41.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.148+0000 7f072d3a0640 1 -- 172.21.15.40:0/1979585991 wait complete. 2024-04-02T16:12:41.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-02T16:12:41.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 Processor -- start 2024-04-02T16:12:41.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 -- start start 2024-04-02T16:12:41.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7370073230 0x7f73700b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73700b19e0 con 0x7f7370148e50 2024-04-02T16:12:41.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73700b05c0 con 0x7f73700b23d0 2024-04-02T16:12:41.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f73771a5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f73700b08a0 con 0x7f7370073230 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f7375742640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7370148e50 0x7f7370149280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34098/0 (socket says 172.21.15.40:34098) 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7370073230 0x7f73700b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f7375742640 1 -- 172.21.15.40:0/1646206776 learned_addr learned my addr 172.21.15.40:0/1646206776 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736f7fe640 1 -- 172.21.15.40:0/1646206776 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611609603 0 0) 0x7f73700b19e0 con 0x7f7370148e50 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736f7fe640 1 -- 172.21.15.40:0/1646206776 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f735c003530 con 0x7f7370148e50 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736ffff640 1 -- 172.21.15.40:0/1646206776 >> v1:172.21.15.204:6789/0 conn(0x7f73700b23d0 legacy=0x7f737009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736ffff640 1 -- 172.21.15.40:0/1646206776 >> v1:172.21.15.40:6789/0 conn(0x7f7370148e50 legacy=0x7f7370149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.280+0000 7f736ffff640 1 -- 172.21.15.40:0/1646206776 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73700b0b80 con 0x7f7370073230 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736ffff640 1 --2- 172.21.15.40:0/1646206776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370073230 0x7f73700b27b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7360009d90 tx=0x7f736001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c078bb67eacf7855 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736f7fe640 1 -- 172.21.15.40:0/1646206776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600b2070 con 0x7f7370073230 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736f7fe640 1 -- 172.21.15.40:0/1646206776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600a58c0 con 0x7f7370073230 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736f7fe640 1 -- 172.21.15.40:0/1646206776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600a5c90 con 0x7f7370073230 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- 172.21.15.40:0/1646206776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370073230 msgr2=0x7f73700b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 --2- 172.21.15.40:0/1646206776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370073230 0x7f73700b27b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7360009d90 tx=0x7f736001c8e0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- 172.21.15.40:0/1646206776 shutdown_connections 2024-04-02T16:12:41.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 --2- 172.21.15.40:0/1646206776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370073230 0x7f73700b27b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- 172.21.15.40:0/1646206776 >> 172.21.15.40:0/1646206776 conn(0x7f73700b8fd0 msgr2=0x7f73700b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- 172.21.15.40:0/1646206776 shutdown_connections 2024-04-02T16:12:41.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- 172.21.15.40:0/1646206776 wait complete. 2024-04-02T16:12:41.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 Processor -- start 2024-04-02T16:12:41.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- start start 2024-04-02T16:12:41.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 0x7f737013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 0x7f73701e0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7370153a30 con 0x7f7370148e50 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73701538b0 con 0x7f73700b23d0 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f73771a5640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7370153bb0 con 0x7f7370073230 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50516/0 (socket says 172.21.15.40:50516) 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 -- 172.21.15.40:0/144887184 learned_addr learned my addr 172.21.15.40:0/144887184 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736ffff640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 0x7f73701e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7374f41640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 0x7f737013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 -- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 msgr2=0x7f737013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 0x7f737013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 -- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 msgr2=0x7f73701e0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 0x7f73701e0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 -- 172.21.15.40:0/144887184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73700b7090 con 0x7f7370148e50 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736ffff640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 0x7f73701e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7374f41640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 0x7f737013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f7375742640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f736800c3c0 tx=0x7f736800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368017070 con 0x7f7370148e50 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368005cf0 con 0x7f7370148e50 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.284+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368007310 con 0x7f7370148e50 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737014e110 con 0x7f7370148e50 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737013fa50 con 0x7f7370148e50 2024-04-02T16:12:41.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73700685f0 con 0x7f7370148e50 2024-04-02T16:12:41.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f736801a070 con 0x7f7370148e50 2024-04-02T16:12:41.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f736d7fa640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 0x7f735006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f7368085b80 con 0x7f7370148e50 2024-04-02T16:12:41.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f7374f41640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 0x7f735006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.288+0000 7f7374f41640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 0x7f735006a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73701e1450 tx=0x7f7358009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.292+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f73680565b0 con 0x7f7370148e50 2024-04-02T16:12:41.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.448+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f73700b1690 con 0x7f7350068380 2024-04-02T16:12:41.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f736d7fa640 1 -- 172.21.15.40:0/144887184 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f73700b1690 con 0x7f7350068380 2024-04-02T16:12:41.452 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-04-02T16:12:41.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 msgr2=0x7f735006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 0x7f735006a840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f73701e1450 tx=0x7f7358009040 comp rx=0 tx=0).stop 2024-04-02T16:12:41.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 msgr2=0x7f73701e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f736800c3c0 tx=0x7f736800c880 comp rx=0 tx=0).stop 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 shutdown_connections 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f7350068380 0x7f735006a840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7370148e50 0x7f73701e0860 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73700b23d0 0x7f73701e0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 --2- 172.21.15.40:0/144887184 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7370073230 0x7f737013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 >> 172.21.15.40:0/144887184 conn(0x7f73700b8fd0 msgr2=0x7f737006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 shutdown_connections 2024-04-02T16:12:41.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.452+0000 7f73771a5640 1 -- 172.21.15.40:0/144887184 wait complete. 2024-04-02T16:12:41.471 INFO:tasks.ceph:Scrubbing osd.1 2024-04-02T16:12:41.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 Processor -- start 2024-04-02T16:12:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 -- start start 2024-04-02T16:12:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb980b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb980b5320 0x7feb98072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feb980730a0 0x7feb980b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7feb980b2c40 con 0x7feb980b5320 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7feb980b2560 con 0x7feb980730a0 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9d09d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb980b2f00 con 0x7feb9806d830 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb980b5320 0x7feb98072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feb980b5320 0x7feb98072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50532/0 (socket says 172.21.15.40:50532) 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 -- 172.21.15.40:0/1272275236 learned_addr learned my addr 172.21.15.40:0/1272275236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb96d9d640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb980b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9759e640 1 --2- 172.21.15.40:0/1272275236 >> v2:172.21.15.204:3300/0 conn(0x7feb980730a0 0x7feb980b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 -- 172.21.15.40:0/1272275236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 msgr2=0x7feb980b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb980b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 -- 172.21.15.40:0/1272275236 >> v2:172.21.15.204:3300/0 conn(0x7feb980730a0 msgr2=0x7feb980b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 --2- 172.21.15.40:0/1272275236 >> v2:172.21.15.204:3300/0 conn(0x7feb980730a0 0x7feb980b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.592+0000 7feb9659c640 1 -- 172.21.15.40:0/1272275236 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb980b4ec0 con 0x7feb980b5320 2024-04-02T16:12:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9659c640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980b5320 0x7feb98072b60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feb980b57a0 tx=0x7feb8001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f95d7c22ee2c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb95d9b640 1 -- 172.21.15.40:0/1272275236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800b0070 con 0x7feb980b5320 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb95d9b640 1 -- 172.21.15.40:0/1272275236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb800ab600 con 0x7feb980b5320 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb95d9b640 1 -- 172.21.15.40:0/1272275236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800ab9d0 con 0x7feb980b5320 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- 172.21.15.40:0/1272275236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980b5320 msgr2=0x7feb98072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980b5320 0x7feb98072b60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7feb980b57a0 tx=0x7feb8001c260 comp rx=0 tx=0).stop 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- 172.21.15.40:0/1272275236 shutdown_connections 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 --2- 172.21.15.40:0/1272275236 >> v2:172.21.15.204:3300/0 conn(0x7feb980730a0 0x7feb980b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980b5320 0x7feb98072b60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 --2- 172.21.15.40:0/1272275236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb980b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- 172.21.15.40:0/1272275236 >> 172.21.15.40:0/1272275236 conn(0x7feb980b5890 msgr2=0x7feb980b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- 172.21.15.40:0/1272275236 shutdown_connections 2024-04-02T16:12:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- 172.21.15.40:0/1272275236 wait complete. 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 Processor -- start 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.596+0000 7feb9d09d640 1 -- start start 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb98072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 0x7feb9806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb98139b00 0x7feb98134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7feb98145470 con 0x7feb980730a0 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb980b27b0 con 0x7feb98139b00 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feb981455f0 con 0x7feb9806d830 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 0x7feb9806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 0x7feb9806f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50536/0 (socket says 172.21.15.40:50536) 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 -- 172.21.15.40:0/824554803 learned_addr learned my addr 172.21.15.40:0/824554803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9759e640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb98139b00 0x7feb98134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb96d9d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb98072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 -- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 msgr2=0x7feb98072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb9806d830 0x7feb98072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 -- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb98139b00 msgr2=0x7feb98134f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb98139b00 0x7feb98134f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9814f890 con 0x7feb980730a0 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9659c640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 0x7feb9806f9a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feb8001cb00 tx=0x7feb80019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800bb070 con 0x7feb980730a0 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb80004230 con 0x7feb980730a0 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb980718a0 con 0x7feb980730a0 2024-04-02T16:12:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb800b0040 con 0x7feb980730a0 2024-04-02T16:12:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb981421a0 con 0x7feb980730a0 2024-04-02T16:12:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb980685f0 con 0x7feb980730a0 2024-04-02T16:12:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7feb800be070 con 0x7feb980730a0 2024-04-02T16:12:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feb78068380 0x7feb7806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7feb80120570 con 0x7feb980730a0 2024-04-02T16:12:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7feb7806e240 0x7feb78070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb800b4e50 con 0x7feb7806e240 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb80120920 con 0x7feb980730a0 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb96d9d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feb78068380 0x7feb7806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.600+0000 7feb9759e640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7feb7806e240 0x7feb78070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.604+0000 7feb9759e640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7feb7806e240 0x7feb78070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.604+0000 7feb96d9d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feb78068380 0x7feb7806a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb88003fd0 tx=0x7feb880039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.604+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== osd.1 v2:172.21.15.40:6816/422638552 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb800b4e50 con 0x7feb7806e240 2024-04-02T16:12:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 --> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feb9806fee0 con 0x7feb7806e240 2024-04-02T16:12:41.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb8f7fe640 1 -- 172.21.15.40:0/824554803 <== osd.1 v2:172.21.15.40:6816/422638552 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feb9806fee0 con 0x7feb7806e240 2024-04-02T16:12:41.618 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:41.618 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:41.619 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7feb7806e240 msgr2=0x7feb78070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:6816/422638552,v1:172.21.15.40:6817/422638552] conn(0x7feb7806e240 0x7feb78070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feb78068380 msgr2=0x7feb7806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7feb78068380 0x7feb7806a840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feb88003fd0 tx=0x7feb880039a0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 msgr2=0x7feb9806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 --2- 172.21.15.40:0/824554803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb980730a0 0x7feb9806f9a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feb8001cb00 tx=0x7feb80019000 comp rx=0 tx=0).stop 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9759e640 1 -- 172.21.15.40:0/824554803 reap_dead start 2024-04-02T16:12:41.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 shutdown_connections 2024-04-02T16:12:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 >> 172.21.15.40:0/824554803 conn(0x7feb980b5890 msgr2=0x7feb980b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 shutdown_connections 2024-04-02T16:12:41.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.616+0000 7feb9d09d640 1 -- 172.21.15.40:0/824554803 wait complete. 2024-04-02T16:12:41.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-02T16:12:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 Processor -- start 2024-04-02T16:12:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 -- start start 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8da00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8da00b19e0 con 0x7f8da00b23d0 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8da00b05c0 con 0x7f8da0073230 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da6995640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8da00b08a0 con 0x7f8da0149560 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8da00b23d0 0x7f8da009daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34110/0 (socket says 172.21.15.40:34110) 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 -- 172.21.15.40:0/2524473529 learned_addr learned my addr 172.21.15.40:0/2524473529 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da4f32640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8da00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194506184 0 0) 0x7f8da00b19e0 con 0x7f8da00b23d0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d88003530 con 0x7f8da00b23d0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2612422021 0 0) 0x7f8d88003530 con 0x7f8da00b23d0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 msgr2=0x7f8da00b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8da00b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 >> v1:172.21.15.204:6789/0 conn(0x7f8da0073230 legacy=0x7f8da0148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da00b0b80 con 0x7f8da00b23d0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f8d90001680 con 0x7f8da00b23d0 2024-04-02T16:12:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 >> v1:172.21.15.40:6789/0 conn(0x7f8da00b23d0 legacy=0x7f8da009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d880039f0 0x7f8d88003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88004810 0x7f8d88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8d88005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8da00b08a0 con 0x7f8d880039f0 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8800bb30 con 0x7f8d88004810 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8800bd30 con 0x7f8da0149560 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d880039f0 0x7f8d88003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9f7fe640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88004810 0x7f8d88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da4f32640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8d88005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 -- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 msgr2=0x7f8d88005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8d88005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 -- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88004810 msgr2=0x7f8d88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88004810 0x7f8d88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9ffff640 1 -- 172.21.15.40:0/2524473529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d88003530 con 0x7f8d880039f0 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8da4f32640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da0149560 0x7f8d88005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.744+0000 7f8d9f7fe640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88004810 0x7f8d88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d880039f0 0x7f8d88003e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000c210 tx=0x7f8d9001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc33bc50805502b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90007d40 con 0x7f8d880039f0 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d900a9070 con 0x7f8d880039f0 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8d9effd640 1 -- 172.21.15.40:0/2524473529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90005d40 con 0x7f8d880039f0 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- 172.21.15.40:0/2524473529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d880039f0 msgr2=0x7f8d88003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 --2- 172.21.15.40:0/2524473529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d880039f0 0x7f8d88003e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000c210 tx=0x7f8d9001cca0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da4f32640 1 -- 172.21.15.40:0/2524473529 reap_dead start 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- 172.21.15.40:0/2524473529 shutdown_connections 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- 172.21.15.40:0/2524473529 >> 172.21.15.40:0/2524473529 conn(0x7f8da00b8fd0 msgr2=0x7f8da00b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- 172.21.15.40:0/2524473529 shutdown_connections 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- 172.21.15.40:0/2524473529 wait complete. 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 Processor -- start 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.748+0000 7f8da6995640 1 -- start start 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d880039f0 0x7f8da0142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da01e1dd0 0x7f8da01e2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0151a20 con 0x7f8da01404d0 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01518a0 con 0x7f8d880039f0 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8da0151ba0 con 0x7f8da01e1dd0 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50560/0 (socket says 172.21.15.40:50560) 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 -- 172.21.15.40:0/3692051871 learned_addr learned my addr 172.21.15.40:0/3692051871 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:41.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d880039f0 0x7f8da0142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da4f32640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da01e1dd0 0x7f8da01e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 -- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da01e1dd0 msgr2=0x7f8da01e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da01e1dd0 0x7f8da01e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 -- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d880039f0 msgr2=0x7f8da0142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d880039f0 0x7f8da0142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 -- 172.21.15.40:0/3692051871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da0140e90 con 0x7f8da01404d0 2024-04-02T16:12:41.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9f7fe640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400d4c0 tx=0x7f8d9400d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94005480 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d94017070 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da0146310 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003840 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da0141ff0 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da0140100 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8d940039e0 con 0x7f8da01404d0 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 0x7f8d8c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:41.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f8d94086410 con 0x7f8da01404d0 2024-04-02T16:12:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.752+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 0x7f8d8c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.756+0000 7f8d9ffff640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 0x7f8d8c06a870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d900ab000 tx=0x7f8d900b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:41.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.756+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f8da0140e90 con 0x7f8da01404d0 2024-04-02T16:12:41.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.920+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f8da00685f0 con 0x7f8d8c0683b0 2024-04-02T16:12:41.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8d9cff9640 1 -- 172.21.15.40:0/3692051871 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8da00685f0 con 0x7f8d8c0683b0 2024-04-02T16:12:41.924 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-04-02T16:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 msgr2=0x7f8d8c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 0x7f8d8c06a870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d900ab000 tx=0x7f8d900b1040 comp rx=0 tx=0).stop 2024-04-02T16:12:41.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 msgr2=0x7f8da0140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400d4c0 tx=0x7f8d9400d980 comp rx=0 tx=0).stop 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 shutdown_connections 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8da01e1dd0 0x7f8da01e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f8d8c0683b0 0x7f8d8c06a870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8da01404d0 0x7f8da0140950 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 --2- 172.21.15.40:0/3692051871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d880039f0 0x7f8da0142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 >> 172.21.15.40:0/3692051871 conn(0x7f8da00b8fd0 msgr2=0x7f8da006cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:41.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 shutdown_connections 2024-04-02T16:12:41.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:41.924+0000 7f8da6995640 1 -- 172.21.15.40:0/3692051871 wait complete. 2024-04-02T16:12:41.943 INFO:tasks.ceph:Scrubbing osd.2 2024-04-02T16:12:41.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:42.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 Processor -- start 2024-04-02T16:12:42.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 -- start start 2024-04-02T16:12:42.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 0x7f5b3814ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b380b4c40 con 0x7f5b3814b2a0 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b380b4160 con 0x7f5b381419a0 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3e8bb640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3802c910 con 0x7f5b38141d80 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3ce58640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5b3814b2a0 0x7f5b38152100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34118/0 (socket says 172.21.15.40:34118) 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b3ce58640 1 -- 172.21.15.40:0/1038989591 learned_addr learned my addr 172.21.15.40:0/1038989591 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 0x7f5b3814ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660352701 0 0) 0x7f5b380b4c40 con 0x7f5b3814b2a0 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b20003530 con 0x7f5b3814b2a0 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1927915724 0 0) 0x7f5b20003530 con 0x7f5b3814b2a0 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 msgr2=0x7f5b3814ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 0x7f5b3814ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 >> v1:172.21.15.204:6789/0 conn(0x7f5b381419a0 legacy=0x7f5b3802c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b380b4440 con 0x7f5b3814b2a0 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f5b30001a00 con 0x7f5b3814b2a0 2024-04-02T16:12:42.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 >> v1:172.21.15.40:6789/0 conn(0x7f5b3814b2a0 legacy=0x7f5b38152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b200039f0 0x7f5b20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 0x7f5b200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3802c910 con 0x7f5b20004810 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2000bb10 con 0x7f5b200039f0 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2000bd10 con 0x7f5b38141d80 2024-04-02T16:12:42.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 -- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 msgr2=0x7f5b200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b38141d80 0x7f5b200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 -- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b200039f0 msgr2=0x7f5b20003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b200039f0 0x7f5b20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.064+0000 7f5b377fe640 1 -- 172.21.15.40:0/1038989591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b20003530 con 0x7f5b20004810 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b377fe640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b20004c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b38142180 tx=0x7f5b2c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43eeae76c6465e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0b2070 con 0x7f5b20004810 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0a5600 con 0x7f5b20004810 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b36ffd640 1 -- 172.21.15.40:0/1038989591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c009690 con 0x7f5b20004810 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/1038989591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 msgr2=0x7f5b20004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 --2- 172.21.15.40:0/1038989591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b20004c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b38142180 tx=0x7f5b2c01ca10 comp rx=0 tx=0).stop 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3ce58640 1 -- 172.21.15.40:0/1038989591 reap_dead start 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/1038989591 shutdown_connections 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/1038989591 >> 172.21.15.40:0/1038989591 conn(0x7f5b3809b370 msgr2=0x7f5b3809b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/1038989591 shutdown_connections 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/1038989591 wait complete. 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 Processor -- start 2024-04-02T16:12:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- start start 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b3809c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b381499b0 0x7f5b38149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b3809c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b3809c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50572/0 (socket says 172.21.15.40:50572) 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b3809dd80 0x7f5b3809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380702a0 con 0x7f5b20004810 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b37fff640 1 -- 172.21.15.40:0/281768899 learned_addr learned my addr 172.21.15.40:0/281768899 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380b5b10 con 0x7f5b381499b0 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.068+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b380b5df0 con 0x7f5b3809dd80 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 -- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b3809dd80 msgr2=0x7f5b3809e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b3809dd80 0x7f5b3809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 -- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b381499b0 msgr2=0x7f5b38149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b381499b0 0x7f5b38149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b381dd450 con 0x7f5b20004810 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b37fff640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b3809c920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c4d0 tx=0x7f5b2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280052d0 con 0x7f5b20004810 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b28005490 con 0x7f5b20004810 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28012850 con 0x7f5b20004810 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b38142250 con 0x7f5b20004810 2024-04-02T16:12:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b381dd9f0 con 0x7f5b20004810 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b381dddd0 con 0x7f5b20004810 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f5b280129f0 con 0x7f5b20004810 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5b240683b0 0x7f5b2406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f5b28086ce0 con 0x7f5b20004810 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f5b2406e270 0x7f5b24070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b28003e50 con 0x7f5b2406e270 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5b28087170 con 0x7f5b20004810 2024-04-02T16:12:42.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b3ce58640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f5b2406e270 0x7f5b24070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b377fe640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5b240683b0 0x7f5b2406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b3ce58640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f5b2406e270 0x7f5b24070690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== osd.2 v2:172.21.15.40:6824/658893798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b28003e50 con 0x7f5b2406e270 2024-04-02T16:12:42.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.072+0000 7f5b377fe640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5b240683b0 0x7f5b2406a870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0af000 tx=0x7f5b2c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 --> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5b380685f0 con 0x7f5b2406e270 2024-04-02T16:12:42.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b34ff9640 1 -- 172.21.15.40:0/281768899 <== osd.2 v2:172.21.15.40:6824/658893798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5b380685f0 con 0x7f5b2406e270 2024-04-02T16:12:42.088 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:42.088 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:42.088 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:42.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f5b2406e270 msgr2=0x7f5b24070690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:6824/658893798,v1:172.21.15.40:6825/658893798] conn(0x7f5b2406e270 0x7f5b24070690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5b240683b0 msgr2=0x7f5b2406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f5b240683b0 0x7f5b2406a870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0af000 tx=0x7f5b2c004740 comp rx=0 tx=0).stop 2024-04-02T16:12:42.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 msgr2=0x7f5b3809c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 --2- 172.21.15.40:0/281768899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b20004810 0x7f5b3809c920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c4d0 tx=0x7f5b2800c990 comp rx=0 tx=0).stop 2024-04-02T16:12:42.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3ce58640 1 -- 172.21.15.40:0/281768899 reap_dead start 2024-04-02T16:12:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 shutdown_connections 2024-04-02T16:12:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 >> 172.21.15.40:0/281768899 conn(0x7f5b3809b370 msgr2=0x7f5b380b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 shutdown_connections 2024-04-02T16:12:42.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.088+0000 7f5b3e8bb640 1 -- 172.21.15.40:0/281768899 wait complete. 2024-04-02T16:12:42.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-02T16:12:42.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 Processor -- start 2024-04-02T16:12:42.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 -- start start 2024-04-02T16:12:42.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ca8151830 0x7f9ca8153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca80af670 con 0x7f9ca8154160 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca80af950 con 0x7f9ca80ae320 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9caeccc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca80aff10 con 0x7f9ca8151830 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ca8151830 0x7f9ca8153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9cad269640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9ca8154160 0x7f9ca8156550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34120/0 (socket says 172.21.15.40:34120) 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9cad269640 1 -- 172.21.15.40:0/3162966785 learned_addr learned my addr 172.21.15.40:0/3162966785 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca7fff640 1 -- 172.21.15.40:0/3162966785 >> v1:172.21.15.204:6789/0 conn(0x7f9ca80ae320 legacy=0x7f9ca806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca7fff640 1 -- 172.21.15.40:0/3162966785 >> v1:172.21.15.40:6789/0 conn(0x7f9ca8154160 legacy=0x7f9ca8156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca7fff640 1 -- 172.21.15.40:0/3162966785 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80b5260 con 0x7f9ca8151830 2024-04-02T16:12:42.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca77fe640 1 -- 172.21.15.40:0/3162966785 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787678075 0 0) 0x7f9ca80af670 con 0x7f9ca8154160 2024-04-02T16:12:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca7fff640 1 --2- 172.21.15.40:0/3162966785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca8153c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097c0 tx=0x7f9c9c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bf4596cbe7be20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca77fe640 1 -- 172.21.15.40:0/3162966785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b0070 con 0x7f9ca8151830 2024-04-02T16:12:42.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca77fe640 1 -- 172.21.15.40:0/3162966785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c0ab600 con 0x7f9ca8151830 2024-04-02T16:12:42.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.212+0000 7f9ca77fe640 1 -- 172.21.15.40:0/3162966785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0ab9d0 con 0x7f9ca8151830 2024-04-02T16:12:42.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/3162966785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 msgr2=0x7f9ca8153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 --2- 172.21.15.40:0/3162966785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca8153c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0097c0 tx=0x7f9c9c01c310 comp rx=0 tx=0).stop 2024-04-02T16:12:42.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/3162966785 shutdown_connections 2024-04-02T16:12:42.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 --2- 172.21.15.40:0/3162966785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca8153c20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/3162966785 >> 172.21.15.40:0/3162966785 conn(0x7f9ca80b8fd0 msgr2=0x7f9ca80b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/3162966785 shutdown_connections 2024-04-02T16:12:42.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/3162966785 wait complete. 2024-04-02T16:12:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 Processor -- start 2024-04-02T16:12:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- start start 2024-04-02T16:12:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 0x7f9ca809c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ca8154160 0x7f9ca81e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80af2f0 con 0x7f9ca8151830 2024-04-02T16:12:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80b0300 con 0x7f9ca80ae320 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca80aeef0 con 0x7f9ca8154160 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50582/0 (socket says 172.21.15.40:50582) 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 -- 172.21.15.40:0/725921536 learned_addr learned my addr 172.21.15.40:0/725921536 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caca68640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 0x7f9ca809c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9cad269640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ca8154160 0x7f9ca81e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 -- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ca8154160 msgr2=0x7f9ca81e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ca8154160 0x7f9ca81e7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 -- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 msgr2=0x7f9ca809c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 0x7f9ca809c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 -- 172.21.15.40:0/725921536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8156160 con 0x7f9ca8151830 2024-04-02T16:12:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caca68640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 0x7f9ca809c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca7fff640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae040 tx=0x7f9c9c01ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b0050 con 0x7f9ca8151830 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c0bc070 con 0x7f9ca8151830 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0b5a50 con 0x7f9ca8151830 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca80704a0 con 0x7f9ca8151830 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.216+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca8070780 con 0x7f9ca8151830 2024-04-02T16:12:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca80685f0 con 0x7f9ca8151830 2024-04-02T16:12:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f9c9c0ab4a0 con 0x7f9ca8151830 2024-04-02T16:12:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9ca57fa640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9c9c120ee0 con 0x7f9ca8151830 2024-04-02T16:12:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9caca68640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.220+0000 7f9caca68640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 0x7f9c9406a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9c98003e40 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.224+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f9ca80704a0 con 0x7f9ca8151830 2024-04-02T16:12:42.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.380+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f9ca80b05b0 con 0x7f9c94068380 2024-04-02T16:12:42.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.380+0000 7f9ca57fa640 1 -- 172.21.15.40:0/725921536 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9ca80b05b0 con 0x7f9c94068380 2024-04-02T16:12:42.382 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 msgr2=0x7f9c9406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 0x7f9c9406a840 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9c98003e40 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 msgr2=0x7f9ca81e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0ae040 tx=0x7f9c9c01ab50 comp rx=0 tx=0).stop 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 shutdown_connections 2024-04-02T16:12:42.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ca8154160 0x7f9ca81e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f9c94068380 0x7f9c9406a840 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca8151830 0x7f9ca81e7630 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 --2- 172.21.15.40:0/725921536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ca80ae320 0x7f9ca809c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 >> 172.21.15.40:0/725921536 conn(0x7f9ca80b8fd0 msgr2=0x7f9ca80b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 shutdown_connections 2024-04-02T16:12:42.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.384+0000 7f9caeccc640 1 -- 172.21.15.40:0/725921536 wait complete. 2024-04-02T16:12:42.399 INFO:tasks.ceph:Scrubbing osd.3 2024-04-02T16:12:42.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 Processor -- start 2024-04-02T16:12:42.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 -- start start 2024-04-02T16:12:42.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e080b5940 0x7f3e080b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 0x7f3e080b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e080b8590 con 0x7f3e080b81b0 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3e080b64b0 con 0x7f3e080b5940 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0fc7c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f3e080b5e60 con 0x7f3e081492a0 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0da18640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3e080b81b0 0x7f3e08148b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34124/0 (socket says 172.21.15.40:34124) 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0d217640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e080b5940 0x7f3e080b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0da18640 1 -- 172.21.15.40:0/457093051 learned_addr learned my addr 172.21.15.40:0/457093051 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0e219640 1 --2- 172.21.15.40:0/457093051 >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 0x7f3e080b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0d217640 1 -- 172.21.15.40:0/457093051 >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 msgr2=0x7f3e080b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0d217640 1 --2- 172.21.15.40:0/457093051 >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 0x7f3e080b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0d217640 1 -- 172.21.15.40:0/457093051 >> v1:172.21.15.40:6789/0 conn(0x7f3e080b81b0 legacy=0x7f3e08148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0ca16640 1 -- 172.21.15.40:0/457093051 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557165415 0 0) 0x7f3e080b8590 con 0x7f3e080b81b0 2024-04-02T16:12:42.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0d217640 1 -- 172.21.15.40:0/457093051 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e080b6140 con 0x7f3e080b5940 2024-04-02T16:12:42.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.524+0000 7f3e0e219640 1 --2- 172.21.15.40:0/457093051 >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 0x7f3e080b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:42.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0d217640 1 --2- 172.21.15.40:0/457093051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e080b4a50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e0806ffa0 tx=0x7f3df801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8decae07237a4025 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0ca16640 1 -- 172.21.15.40:0/457093051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80b0070 con 0x7f3e080b5940 2024-04-02T16:12:42.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0ca16640 1 -- 172.21.15.40:0/457093051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df80ab600 con 0x7f3e080b5940 2024-04-02T16:12:42.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0ca16640 1 -- 172.21.15.40:0/457093051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80ab9d0 con 0x7f3e080b5940 2024-04-02T16:12:42.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/457093051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 msgr2=0x7f3e080b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/457093051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e080b4a50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e0806ffa0 tx=0x7f3df801c3f0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/457093051 shutdown_connections 2024-04-02T16:12:42.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/457093051 >> v2:172.21.15.204:3300/0 conn(0x7f3e081492a0 0x7f3e080b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/457093051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e080b4a50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/457093051 >> 172.21.15.40:0/457093051 conn(0x7f3e0802c5d0 msgr2=0x7f3e0802c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/457093051 shutdown_connections 2024-04-02T16:12:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/457093051 wait complete. 2024-04-02T16:12:42.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 Processor -- start 2024-04-02T16:12:42.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- start start 2024-04-02T16:12:42.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e0806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e080b81b0 0x7f3e081e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e081492a0 0x7f3e081e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3e081527f0 con 0x7f3e080b5940 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e080b76b0 con 0x7f3e080b81b0 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.528+0000 7f3e0fc7c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f3e08152970 con 0x7f3e081492a0 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e0806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e0806dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50586/0 (socket says 172.21.15.40:50586) 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 -- 172.21.15.40:0/3355991046 learned_addr learned my addr 172.21.15.40:0/3355991046 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0d217640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e080b81b0 0x7f3e081e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 -- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e081492a0 msgr2=0x7f3e081e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e081492a0 0x7f3e081e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 -- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e080b81b0 msgr2=0x7f3e081e0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e080b81b0 0x7f3e081e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e081405d0 con 0x7f3e080b5940 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0da18640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e0806dc70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e040011f0 tx=0x7f3e0400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04018070 con 0x7f3e080b5940 2024-04-02T16:12:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e040074e0 con 0x7f3e080b5940 2024-04-02T16:12:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04004a10 con 0x7f3e080b5940 2024-04-02T16:12:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081e34d0 con 0x7f3e080b5940 2024-04-02T16:12:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e08146680 con 0x7f3e080b5940 2024-04-02T16:12:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e081dfde0 con 0x7f3e080b5940 2024-04-02T16:12:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f3e0401b070 con 0x7f3e080b5940 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3dec068380 0x7f3dec06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3e04085ff0 con 0x7f3e080b5940 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f3dec06e240 0x7f3dec070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dec070d10 con 0x7f3dec06e240 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e04086480 con 0x7f3e080b5940 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0e219640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f3dec06e240 0x7f3dec070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0d217640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3dec068380 0x7f3dec06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0e219640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f3dec06e240 0x7f3dec070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.532+0000 7f3e0d217640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3dec068380 0x7f3dec06a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ad000 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.536+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== osd.3 v2:172.21.15.40:6800/1014552006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dec070d10 con 0x7f3dec06e240 2024-04-02T16:12:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 --> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3e08068d00 con 0x7f3dec06e240 2024-04-02T16:12:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3dfe7fc640 1 -- 172.21.15.40:0/3355991046 <== osd.3 v2:172.21.15.40:6800/1014552006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3e08068d00 con 0x7f3dec06e240 2024-04-02T16:12:42.549 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:42.549 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:42.549 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f3dec06e240 msgr2=0x7f3dec070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:6800/1014552006,v1:172.21.15.40:6802/1014552006] conn(0x7f3dec06e240 0x7f3dec070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3dec068380 msgr2=0x7f3dec06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f3dec068380 0x7f3dec06a840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3df80ad000 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 msgr2=0x7f3e0806dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 --2- 172.21.15.40:0/3355991046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e080b5940 0x7f3e0806dc70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e040011f0 tx=0x7f3e0400c9b0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0e219640 1 -- 172.21.15.40:0/3355991046 reap_dead start 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 shutdown_connections 2024-04-02T16:12:42.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 >> 172.21.15.40:0/3355991046 conn(0x7f3e0802c5d0 msgr2=0x7f3e08137ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 shutdown_connections 2024-04-02T16:12:42.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.548+0000 7f3e0fc7c640 1 -- 172.21.15.40:0/3355991046 wait complete. 2024-04-02T16:12:42.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-02T16:12:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 Processor -- start 2024-04-02T16:12:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 -- start start 2024-04-02T16:12:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 0x7f27e80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e80b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e80b6310 con 0x7f27e80b3500 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f27e80b55d0 con 0x7f27e80b4bf0 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.680+0000 7f27ee757640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27e814b560 con 0x7f27e80b8c80 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27e7fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 0x7f27e80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27eccf4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e80b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27e77fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f27e80b3500 0x7f27e814ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:57534/0 (socket says 172.21.15.40:57534) 2024-04-02T16:12:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27e77fe640 1 -- 172.21.15.40:0/1720832092 learned_addr learned my addr 172.21.15.40:0/1720832092 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27eccf4640 1 -- 172.21.15.40:0/1720832092 >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 msgr2=0x7f27e80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27eccf4640 1 --2- 172.21.15.40:0/1720832092 >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 0x7f27e80b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27eccf4640 1 -- 172.21.15.40:0/1720832092 >> v1:172.21.15.204:6789/0 conn(0x7f27e80b3500 legacy=0x7f27e814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27eccf4640 1 -- 172.21.15.40:0/1720832092 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80b3fa0 con 0x7f27e80b8c80 2024-04-02T16:12:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.684+0000 7f27e7fff640 1 --2- 172.21.15.40:0/1720832092 >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 0x7f27e80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:12:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27eccf4640 1 --2- 172.21.15.40:0/1720832092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e80b90f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a32e0c4fbbb1aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1720832092 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e00b0070 con 0x7f27e80b8c80 2024-04-02T16:12:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1720832092 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e00ab600 con 0x7f27e80b8c80 2024-04-02T16:12:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27e6ffd640 1 -- 172.21.15.40:0/1720832092 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab9d0 con 0x7f27e80b8c80 2024-04-02T16:12:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- 172.21.15.40:0/1720832092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 msgr2=0x7f27e80b90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- 172.21.15.40:0/1720832092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e80b90f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e001c380 comp rx=0 tx=0).stop 2024-04-02T16:12:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- 172.21.15.40:0/1720832092 shutdown_connections 2024-04-02T16:12:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- 172.21.15.40:0/1720832092 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e80b90f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- 172.21.15.40:0/1720832092 >> v2:172.21.15.204:3300/0 conn(0x7f27e80b4bf0 0x7f27e80b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- 172.21.15.40:0/1720832092 >> 172.21.15.40:0/1720832092 conn(0x7f27e80734b0 msgr2=0x7f27e80bd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- 172.21.15.40:0/1720832092 shutdown_connections 2024-04-02T16:12:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- 172.21.15.40:0/1720832092 wait complete. 2024-04-02T16:12:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 Processor -- start 2024-04-02T16:12:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- start start 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e80b4bf0 0x7f27e81e3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e8146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8154c10 con 0x7f27e80b3500 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8154d90 con 0x7f27e80b4bf0 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.688+0000 7f27ee757640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f27e8154f10 con 0x7f27e80b8c80 2024-04-02T16:12:42.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50602/0 (socket says 172.21.15.40:50602) 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 -- 172.21.15.40:0/2211856201 learned_addr learned my addr 172.21.15.40:0/2211856201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e77fe640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e80b4bf0 0x7f27e81e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27eccf4640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e8146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 -- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 msgr2=0x7f27e8146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e8146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 -- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e80b4bf0 msgr2=0x7f27e81e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e80b4bf0 0x7f27e81e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 -- 172.21.15.40:0/2211856201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e81e3150 con 0x7f27e80b3500 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e7fff640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f27d000ebe0 tx=0x7f27d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0017070 con 0x7f27e80b3500 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e813b740 con 0x7f27e80b3500 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d000cc30 con 0x7f27e80b3500 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0013470 con 0x7f27e80b3500 2024-04-02T16:12:42.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e8145980 con 0x7f27e80b3500 2024-04-02T16:12:42.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e80685f0 con 0x7f27e80b3500 2024-04-02T16:12:42.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f27d001a070 con 0x7f27e80b3500 2024-04-02T16:12:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f27d0086d40 con 0x7f27e80b3500 2024-04-02T16:12:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.692+0000 7f27e77fe640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:42.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.696+0000 7f27e77fe640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 0x7f27c806a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27e8145100 tx=0x7f27d8001a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:42.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.696+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f27d0057040 con 0x7f27e80b3500 2024-04-02T16:12:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.852+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f27e80b3bc0 con 0x7f27c8068380 2024-04-02T16:12:42.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27e4ff9640 1 -- 172.21.15.40:0/2211856201 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f27e80b3bc0 con 0x7f27c8068380 2024-04-02T16:12:42.856 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 msgr2=0x7f27c806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 0x7f27c806a840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27e8145100 tx=0x7f27d8001a40 comp rx=0 tx=0).stop 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 msgr2=0x7f27e8147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f27d000ebe0 tx=0x7f27d000c520 comp rx=0 tx=0).stop 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 shutdown_connections 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e80b8c80 0x7f27e8146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f27c8068380 0x7f27c806a840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e80b4bf0 0x7f27e81e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 --2- 172.21.15.40:0/2211856201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27e80b3500 0x7f27e8147a30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 >> 172.21.15.40:0/2211856201 conn(0x7f27e80734b0 msgr2=0x7f27e8140470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 shutdown_connections 2024-04-02T16:12:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:42.856+0000 7f27ee757640 1 -- 172.21.15.40:0/2211856201 wait complete. 2024-04-02T16:12:42.874 INFO:tasks.ceph:Scrubbing osd.4 2024-04-02T16:12:42.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 Processor -- start 2024-04-02T16:12:43.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 -- start start 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f809c06bef0 0x7f809c0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f809c0afe00 con 0x7f809c0b0500 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f809c06c2d0 con 0x7f809c144ff0 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f80a1c75640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f809c0b0070 con 0x7f809c06bef0 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f8093fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f809c0b0500 0x7f809c1448e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34134/0 (socket says 172.21.15.40:34134) 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f8093fff640 1 -- 172.21.15.40:0/3769159351 learned_addr learned my addr 172.21.15.40:0/3769159351 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809b7fe640 1 --2- 172.21.15.40:0/3769159351 >> v2:172.21.15.204:3300/0 conn(0x7f809c06bef0 0x7f809c0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1284687009 0 0) 0x7f809c0afe00 con 0x7f809c0b0500 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8088003530 con 0x7f809c0b0500 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3798538582 0 0) 0x7f8088003530 con 0x7f809c0b0500 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 >> v2:172.21.15.204:3300/0 conn(0x7f809c06bef0 msgr2=0x7f809c0b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 --2- 172.21.15.40:0/3769159351 >> v2:172.21.15.204:3300/0 conn(0x7f809c06bef0 0x7f809c0b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 >> v1:172.21.15.204:6789/0 conn(0x7f809c144ff0 legacy=0x7f809c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c0ae300 con 0x7f809c0b0500 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7f8084001680 con 0x7f809c0b0500 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 >> v1:172.21.15.40:6789/0 conn(0x7f809c0b0500 legacy=0x7f809c1448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80880039c0 0x7f8088003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8088004810 0x7f8088004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 0x7f808800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0b0070 con 0x7f80880051d0 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f808800c070 con 0x7f80880039c0 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f808800c270 con 0x7f8088004810 2024-04-02T16:12:43.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 0x7f808800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809b7fe640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80880039c0 0x7f8088003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f8093fff640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8088004810 0x7f8088004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 -- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8088004810 msgr2=0x7f8088004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8088004810 0x7f8088004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 -- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80880039c0 msgr2=0x7f8088003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80880039c0 0x7f8088003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.000+0000 7f809bfff640 1 -- 172.21.15.40:0/3769159351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088003530 con 0x7f80880051d0 2024-04-02T16:12:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f809bfff640 1 -- 172.21.15.40:0/3769159351 reap_dead start 2024-04-02T16:12:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f809bfff640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 0x7f808800b9d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8094001e70 tx=0x7f809401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f856a2369e10981f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094001a90 con 0x7f80880051d0 2024-04-02T16:12:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094003ba0 con 0x7f80880051d0 2024-04-02T16:12:43.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f809affd640 1 -- 172.21.15.40:0/3769159351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094005020 con 0x7f80880051d0 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 msgr2=0x7f808800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 0x7f808800b9d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8094001e70 tx=0x7f809401c780 comp rx=0 tx=0).stop 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- 172.21.15.40:0/3769159351 shutdown_connections 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 --2- 172.21.15.40:0/3769159351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80880051d0 0x7f808800b9d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- 172.21.15.40:0/3769159351 >> 172.21.15.40:0/3769159351 conn(0x7f809c0b9d00 msgr2=0x7f809c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- 172.21.15.40:0/3769159351 shutdown_connections 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- 172.21.15.40:0/3769159351 wait complete. 2024-04-02T16:12:43.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 Processor -- start 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.004+0000 7f80a1c75640 1 -- start start 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80880051d0 0x7f809c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 0x7f809c0b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f809c0b33e0 0x7f809c1e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0ae940 con 0x7f809c0b20e0 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0adbd0 con 0x7f809c0b33e0 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f809c0aec20 con 0x7f80880051d0 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 0x7f809c0b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 0x7f809c0b2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50626/0 (socket says 172.21.15.40:50626) 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 -- 172.21.15.40:0/3232217999 learned_addr learned my addr 172.21.15.40:0/3232217999 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f809bfff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f809c0b33e0 0x7f809c1e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 -- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80880051d0 msgr2=0x7f809c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80880051d0 0x7f809c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 -- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f809c0b33e0 msgr2=0x7f809c1e03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f809c0b33e0 0x7f809c1e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c0b3870 con 0x7f809c0b20e0 2024-04-02T16:12:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8093fff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 0x7f809c0b2ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8084007fd0 tx=0x7f808400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084019070 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80840072d0 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80840047f0 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c0b25c0 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c146c50 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f809c068d00 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f8084004990 con 0x7f809c0b20e0 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f807c068380 0x7f807c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f8084085ea0 con 0x7f809c0b20e0 2024-04-02T16:12:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7f807c06e240 0x7f807c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f807c070d10 con 0x7f807c06e240 2024-04-02T16:12:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80840885a0 con 0x7f809c0b20e0 2024-04-02T16:12:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f809b7fe640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f807c068380 0x7f807c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.008+0000 7f809bfff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7f807c06e240 0x7f807c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.012+0000 7f809b7fe640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f807c068380 0x7f807c06a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f809c143610 tx=0x7f808c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.012+0000 7f809bfff640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7f807c06e240 0x7f807c070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.012+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== osd.4 v2:172.21.15.204:6800/3004039088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f807c070d10 con 0x7f807c06e240 2024-04-02T16:12:43.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.024+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 --> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f809c140840 con 0x7f807c06e240 2024-04-02T16:12:43.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f8098ff9640 1 -- 172.21.15.40:0/3232217999 <== osd.4 v2:172.21.15.204:6800/3004039088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f809c140840 con 0x7f807c06e240 2024-04-02T16:12:43.028 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:43.028 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7f807c06e240 msgr2=0x7f807c070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6800/3004039088,v1:172.21.15.204:6806/3004039088] conn(0x7f807c06e240 0x7f807c070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f807c068380 msgr2=0x7f807c06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f807c068380 0x7f807c06a840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f809c143610 tx=0x7f808c008040 comp rx=0 tx=0).stop 2024-04-02T16:12:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 msgr2=0x7f809c0b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 --2- 172.21.15.40:0/3232217999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f809c0b20e0 0x7f809c0b2ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8084007fd0 tx=0x7f808400db50 comp rx=0 tx=0).stop 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f809bfff640 1 -- 172.21.15.40:0/3232217999 reap_dead start 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 shutdown_connections 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 >> 172.21.15.40:0/3232217999 conn(0x7f809c0b9d00 msgr2=0x7f809c0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 shutdown_connections 2024-04-02T16:12:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.028+0000 7f80a1c75640 1 -- 172.21.15.40:0/3232217999 wait complete. 2024-04-02T16:12:43.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-02T16:12:43.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 Processor -- start 2024-04-02T16:12:43.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 -- start start 2024-04-02T16:12:43.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09dc148550 0x7f09dc0af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 0x7f09dc141840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09dc071550 con 0x7f09dc072910 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc147a70 con 0x7f09dc148550 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e46ef640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc147d50 con 0x7f09dc141460 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09dc148550 0x7f09dc0af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09dc148550 0x7f09dc0af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50632/0 (socket says 172.21.15.40:50632) 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 -- 172.21.15.40:0/2573570046 learned_addr learned my addr 172.21.15.40:0/2573570046 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e2c8c640 1 --2- 172.21.15.40:0/2573570046 >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 0x7f09dc141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 -- 172.21.15.40:0/2573570046 >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 msgr2=0x7f09dc141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 --2- 172.21.15.40:0/2573570046 >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 0x7f09dc141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 -- 172.21.15.40:0/2573570046 >> v1:172.21.15.204:6789/0 conn(0x7f09dc072910 legacy=0x7f09dc070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e1c8a640 1 -- 172.21.15.40:0/2573570046 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc148030 con 0x7f09dc148550 2024-04-02T16:12:43.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.152+0000 7f09e2c8c640 1 --2- 172.21.15.40:0/2573570046 >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 0x7f09dc141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:43.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e1c8a640 1 --2- 172.21.15.40:0/2573570046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc0af1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09dc070ce0 tx=0x7f09cc01ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf18e2a79933720 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e1489640 1 -- 172.21.15.40:0/2573570046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0b4070 con 0x7f09dc148550 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e1489640 1 -- 172.21.15.40:0/2573570046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09cc0a7ac0 con 0x7f09dc148550 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e1489640 1 -- 172.21.15.40:0/2573570046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0af440 con 0x7f09dc148550 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- 172.21.15.40:0/2573570046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 msgr2=0x7f09dc0af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- 172.21.15.40:0/2573570046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc0af1e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09dc070ce0 tx=0x7f09cc01ead0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- 172.21.15.40:0/2573570046 shutdown_connections 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- 172.21.15.40:0/2573570046 >> v2:172.21.15.204:3300/0 conn(0x7f09dc141460 0x7f09dc141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- 172.21.15.40:0/2573570046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc0af1e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- 172.21.15.40:0/2573570046 >> 172.21.15.40:0/2573570046 conn(0x7f09dc0b7f50 msgr2=0x7f09dc0b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- 172.21.15.40:0/2573570046 shutdown_connections 2024-04-02T16:12:43.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- 172.21.15.40:0/2573570046 wait complete. 2024-04-02T16:12:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 Processor -- start 2024-04-02T16:12:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- start start 2024-04-02T16:12:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09dc072910 0x7f09dc13dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09dc141460 0x7f09dc1e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc06bf10 con 0x7f09dc148550 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc06bd90 con 0x7f09dc072910 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e46ef640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc06c090 con 0x7f09dc141460 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50638/0 (socket says 172.21.15.40:50638) 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 -- 172.21.15.40:0/474254535 learned_addr learned my addr 172.21.15.40:0/474254535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e248b640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09dc072910 0x7f09dc13dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 -- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09dc141460 msgr2=0x7f09dc1e36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09dc141460 0x7f09dc1e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 -- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09dc072910 msgr2=0x7f09dc13dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09dc072910 0x7f09dc13dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.156+0000 7f09e2c8c640 1 -- 172.21.15.40:0/474254535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc144780 con 0x7f09dc148550 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e2c8c640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009fa0 tx=0x7f09d800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d801a070 con 0x7f09dc148550 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8003bc0 con 0x7f09dc148550 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc1449f0 con 0x7f09dc148550 2024-04-02T16:12:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004030 con 0x7f09dc148550 2024-04-02T16:12:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc09fb10 con 0x7f09dc148550 2024-04-02T16:12:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc068d00 con 0x7f09dc148550 2024-04-02T16:12:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f09d801d070 con 0x7f09dc148550 2024-04-02T16:12:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 0x7f09c006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f09d80869b0 con 0x7f09dc148550 2024-04-02T16:12:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e248b640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 0x7f09c006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.160+0000 7f09e248b640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 0x7f09c006a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09dc09ff10 tx=0x7f09d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.164+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f09d8010070 con 0x7f09dc148550 2024-04-02T16:12:43.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f09dc0b07e0 con 0x7f09c0068380 2024-04-02T16:12:43.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09d2ffd640 1 -- 172.21.15.40:0/474254535 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09dc0b07e0 con 0x7f09c0068380 2024-04-02T16:12:43.321 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 msgr2=0x7f09c006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 0x7f09c006a840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f09dc09ff10 tx=0x7f09d4009040 comp rx=0 tx=0).stop 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 msgr2=0x7f09dc1e3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009fa0 tx=0x7f09d800d560 comp rx=0 tx=0).stop 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 shutdown_connections 2024-04-02T16:12:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09dc141460 0x7f09dc1e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f09c0068380 0x7f09c006a840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09dc148550 0x7f09dc1e3c30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 --2- 172.21.15.40:0/474254535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09dc072910 0x7f09dc13dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 >> 172.21.15.40:0/474254535 conn(0x7f09dc0b7f50 msgr2=0x7f09dc0b91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 shutdown_connections 2024-04-02T16:12:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.320+0000 7f09e46ef640 1 -- 172.21.15.40:0/474254535 wait complete. 2024-04-02T16:12:43.338 INFO:tasks.ceph:Scrubbing osd.5 2024-04-02T16:12:43.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:43.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 Processor -- start 2024-04-02T16:12:43.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 -- start start 2024-04-02T16:12:43.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7800b59b0 0x7fd780072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7800b1810 con 0x7fd7800b73c0 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7800b77a0 con 0x7fd78014df70 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.464+0000 7fd787986640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd7800b5e60 con 0x7fd7800b59b0 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7800b73c0 0x7fd780145600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34136/0 (socket says 172.21.15.40:34136) 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 -- 172.21.15.40:0/1298842574 learned_addr learned my addr 172.21.15.40:0/1298842574 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd784f21640 1 --2- 172.21.15.40:0/1298842574 >> v2:172.21.15.204:3300/0 conn(0x7fd7800b59b0 0x7fd780072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553058777 0 0) 0x7fd7800b1810 con 0x7fd7800b73c0 2024-04-02T16:12:43.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd76c003530 con 0x7fd7800b73c0 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119704649 0 0) 0x7fd7800b77a0 con 0x7fd78014df70 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7800b1810 con 0x7fd78014df70 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095802042 0 0) 0x7fd76c003530 con 0x7fd7800b73c0 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 >> v2:172.21.15.204:3300/0 conn(0x7fd7800b59b0 msgr2=0x7fd780072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 --2- 172.21.15.40:0/1298842574 >> v2:172.21.15.204:3300/0 conn(0x7fd7800b59b0 0x7fd780072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 >> v1:172.21.15.204:6789/0 conn(0x7fd78014df70 legacy=0x7fd780150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7800b6ee0 con 0x7fd7800b73c0 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1430647931 0 0) 0x7fd77c001600 con 0x7fd7800b73c0 2024-04-02T16:12:43.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 >> v1:172.21.15.40:6789/0 conn(0x7fd7800b73c0 legacy=0x7fd780145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd76c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd76c004810 0x7fd76c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd76c005220 0x7fd76c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7800b5e60 con 0x7fd76c003ba0 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c00c150 con 0x7fd76c004810 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c00c350 con 0x7fd76c005220 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd76c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd784f21640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd76c004810 0x7fd76c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785f23640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd76c005220 0x7fd76c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 -- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd76c005220 msgr2=0x7fd76c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd76c005220 0x7fd76c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 -- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd76c004810 msgr2=0x7fd76c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd76c004810 0x7fd76c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 -- 172.21.15.40:0/1298842574 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c003530 con 0x7fd76c003ba0 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785f23640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd76c005220 0x7fd76c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785f23640 1 -- 172.21.15.40:0/1298842574 reap_dead start 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd785722640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd76c003fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd77c007f90 tx=0x7fd77c01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc01fe909ddb3b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c007bb0 con 0x7fd76c003ba0 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77c0a9070 con 0x7fd76c003ba0 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.468+0000 7fd777fff640 1 -- 172.21.15.40:0/1298842574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c005e40 con 0x7fd76c003ba0 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 msgr2=0x7fd76c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd76c003fa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd77c007f90 tx=0x7fd77c01cb00 comp rx=0 tx=0).stop 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/1298842574 shutdown_connections 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 --2- 172.21.15.40:0/1298842574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd76c003fa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/1298842574 >> 172.21.15.40:0/1298842574 conn(0x7fd7800b7ab0 msgr2=0x7fd7800b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/1298842574 shutdown_connections 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/1298842574 wait complete. 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 Processor -- start 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- start start 2024-04-02T16:12:43.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd7801e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7800b98e0 0x7fd7800b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780141a60 0x7fd780141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd780145b90 con 0x7fd76c003ba0 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd780145a10 con 0x7fd780141a60 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd7800b1fa0 con 0x7fd7800b98e0 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd7801e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd7801e17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50652/0 (socket says 172.21.15.40:50652) 2024-04-02T16:12:43.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 -- 172.21.15.40:0/63182793 learned_addr learned my addr 172.21.15.40:0/63182793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd784f21640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7800b98e0 0x7fd7800b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785f23640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780141a60 0x7fd780141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 -- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7800b98e0 msgr2=0x7fd7800b9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7800b98e0 0x7fd7800b9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 -- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780141a60 msgr2=0x7fd780141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780141a60 0x7fd780141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7800babc0 con 0x7fd76c003ba0 2024-04-02T16:12:43.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd784f21640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7800b98e0 0x7fd7800b9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785f23640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd780141a60 0x7fd780141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd785722640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd7801e17d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd77c01c570 tx=0x7fd76c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c007bb0 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77c001400 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7800ba280 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c005be0 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7800b90e0 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.472+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd780068d00 con 0x7fd76c003ba0 2024-04-02T16:12:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd77c007110 con 0x7fd76c003ba0 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd764068380 0x7fd76406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd77c120310 con 0x7fd76c003ba0 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7fd76406e240 0x7fd764070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd764070d10 con 0x7fd76406e240 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd77c1207a0 con 0x7fd76c003ba0 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd784f21640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd764068380 0x7fd76406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd785f23640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7fd76406e240 0x7fd764070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd784f21640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd764068380 0x7fd76406a840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd770009aa0 tx=0x7fd770008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.476+0000 7fd785f23640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7fd76406e240 0x7fd764070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.480+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== osd.5 v2:172.21.15.204:6801/246318220 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd764070d10 con 0x7fd76406e240 2024-04-02T16:12:43.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 --> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7800b77a0 con 0x7fd76406e240 2024-04-02T16:12:43.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd775ffb640 1 -- 172.21.15.40:0/63182793 <== osd.5 v2:172.21.15.204:6801/246318220 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd7800b77a0 con 0x7fd76406e240 2024-04-02T16:12:43.494 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:43.494 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:43.494 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:43.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7fd76406e240 msgr2=0x7fd764070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6801/246318220,v1:172.21.15.204:6804/246318220] conn(0x7fd76406e240 0x7fd764070660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd764068380 msgr2=0x7fd76406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd764068380 0x7fd76406a840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd770009aa0 tx=0x7fd770008040 comp rx=0 tx=0).stop 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 msgr2=0x7fd7801e17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 --2- 172.21.15.40:0/63182793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd76c003ba0 0x7fd7801e17d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd77c01c570 tx=0x7fd76c005840 comp rx=0 tx=0).stop 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd785f23640 1 -- 172.21.15.40:0/63182793 reap_dead start 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 shutdown_connections 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 >> 172.21.15.40:0/63182793 conn(0x7fd7800b7ab0 msgr2=0x7fd78013edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 shutdown_connections 2024-04-02T16:12:43.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.492+0000 7fd787986640 1 -- 172.21.15.40:0/63182793 wait complete. 2024-04-02T16:12:43.510 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-02T16:12:43.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 Processor -- start 2024-04-02T16:12:43.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 -- start start 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f416c0b1c10 0x7f416c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0b25b0 0x7f416c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c0b0a10 con 0x7f416c0ae080 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f416c0b2a00 con 0x7f416c0b25b0 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4173880640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f416c0b3210 con 0x7f416c0b1c10 2024-04-02T16:12:43.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0b25b0 0x7f416c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0b25b0 0x7f416c1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50658/0 (socket says 172.21.15.40:50658) 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 -- 172.21.15.40:0/3651799623 learned_addr learned my addr 172.21.15.40:0/3651799623 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4170e1b640 1 --2- 172.21.15.40:0/3651799623 >> v2:172.21.15.204:3300/0 conn(0x7f416c0b1c10 0x7f416c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 -- 172.21.15.40:0/3651799623 >> v2:172.21.15.204:3300/0 conn(0x7f416c0b1c10 msgr2=0x7f416c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 --2- 172.21.15.40:0/3651799623 >> v2:172.21.15.204:3300/0 conn(0x7f416c0b1c10 0x7f416c0b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 -- 172.21.15.40:0/3651799623 >> v1:172.21.15.204:6789/0 conn(0x7f416c0ae080 legacy=0x7f416c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:43.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.628+0000 7f4171e1d640 1 -- 172.21.15.40:0/3651799623 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c06db10 con 0x7f416c0b25b0 2024-04-02T16:12:43.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4171e1d640 1 --2- 172.21.15.40:0/3651799623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c1522a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f416c0737a0 tx=0x7f416401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dd1cd515ac1232 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4163fff640 1 -- 172.21.15.40:0/3651799623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640a57a0 con 0x7f416c0b25b0 2024-04-02T16:12:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4163fff640 1 -- 172.21.15.40:0/3651799623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41640a5940 con 0x7f416c0b25b0 2024-04-02T16:12:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4163fff640 1 -- 172.21.15.40:0/3651799623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640a5d10 con 0x7f416c0b25b0 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- 172.21.15.40:0/3651799623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 msgr2=0x7f416c1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 --2- 172.21.15.40:0/3651799623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c1522a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f416c0737a0 tx=0x7f416401c9f0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- 172.21.15.40:0/3651799623 shutdown_connections 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 --2- 172.21.15.40:0/3651799623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c1522a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 --2- 172.21.15.40:0/3651799623 >> v2:172.21.15.204:3300/0 conn(0x7f416c0b1c10 0x7f416c0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- 172.21.15.40:0/3651799623 >> 172.21.15.40:0/3651799623 conn(0x7f416c0b8fd0 msgr2=0x7f416c0b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- 172.21.15.40:0/3651799623 shutdown_connections 2024-04-02T16:12:43.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- 172.21.15.40:0/3651799623 wait complete. 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 Processor -- start 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.632+0000 7f4173880640 1 -- start start 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f416c0ae080 0x7f416c151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 0x7f416c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0ae760 con 0x7f416c0b25b0 2024-04-02T16:12:43.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0afde0 con 0x7f416c0ae080 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0af2f0 con 0x7f416c0b1c10 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50668/0 (socket says 172.21.15.40:50668) 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 -- 172.21.15.40:0/943558402 learned_addr learned my addr 172.21.15.40:0/943558402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4170e1b640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 0x7f416c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f417161c640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f416c0ae080 0x7f416c151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 -- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 msgr2=0x7f416c14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 0x7f416c14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 -- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f416c0ae080 msgr2=0x7f416c151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f416c0ae080 0x7f416c151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 -- 172.21.15.40:0/943558402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c071550 con 0x7f416c0b25b0 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4170e1b640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 0x7f416c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4171e1d640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f416401a620 tx=0x7f4164019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640bc020 con 0x7f416c0b25b0 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41640b88a0 con 0x7f416c0b25b0 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41640b8c90 con 0x7f416c0b25b0 2024-04-02T16:12:43.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c071270 con 0x7f416c0b25b0 2024-04-02T16:12:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c0719e0 con 0x7f416c0b25b0 2024-04-02T16:12:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c0aeb60 con 0x7f416c0b25b0 2024-04-02T16:12:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f41640bf020 con 0x7f416c0b25b0 2024-04-02T16:12:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 0x7f415006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f4164121a40 con 0x7f416c0b25b0 2024-04-02T16:12:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.636+0000 7f417161c640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 0x7f415006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.640+0000 7f417161c640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 0x7f415006a840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f415c001650 tx=0x7f415c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.640+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f41640c6030 con 0x7f416c0b25b0 2024-04-02T16:12:43.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.804+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f416c072940 con 0x7f4150068380 2024-04-02T16:12:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.804+0000 7f4161ffb640 1 -- 172.21.15.40:0/943558402 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f416c072940 con 0x7f4150068380 2024-04-02T16:12:43.807 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-04-02T16:12:43.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 msgr2=0x7f415006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 0x7f415006a840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f415c001650 tx=0x7f415c009040 comp rx=0 tx=0).stop 2024-04-02T16:12:43.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 msgr2=0x7f416c070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f416401a620 tx=0x7f4164019790 comp rx=0 tx=0).stop 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 shutdown_connections 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f416c0b1c10 0x7f416c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f4150068380 0x7f415006a840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0b25b0 0x7f416c070b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 --2- 172.21.15.40:0/943558402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f416c0ae080 0x7f416c151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 >> 172.21.15.40:0/943558402 conn(0x7f416c0b8fd0 msgr2=0x7f416c0b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 shutdown_connections 2024-04-02T16:12:43.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.808+0000 7f4173880640 1 -- 172.21.15.40:0/943558402 wait complete. 2024-04-02T16:12:43.826 INFO:tasks.ceph:Scrubbing osd.6 2024-04-02T16:12:43.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:43.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 Processor -- start 2024-04-02T16:12:43.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 -- start start 2024-04-02T16:12:43.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a080aeb80 0x7f4a08147f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a080b3220 con 0x7f4a0806d830 2024-04-02T16:12:43.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0806dc10 con 0x7f4a08148450 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a0e71e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4a080b13e0 con 0x7f4a080aeb80 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a080aeb80 0x7f4a08147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a080aeb80 0x7f4a08147f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50680/0 (socket says 172.21.15.40:50680) 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a07fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4a0806d830 0x7f4a080ae470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34150/0 (socket says 172.21.15.40:34150) 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 -- 172.21.15.40:0/4058892486 learned_addr learned my addr 172.21.15.40:0/4058892486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 -- 172.21.15.40:0/4058892486 >> v1:172.21.15.204:6789/0 conn(0x7f4a08148450 legacy=0x7f4a0814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 -- 172.21.15.40:0/4058892486 >> v1:172.21.15.40:6789/0 conn(0x7f4a0806d830 legacy=0x7f4a080ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 -- 172.21.15.40:0/4058892486 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a080b52d0 con 0x7f4a080aeb80 2024-04-02T16:12:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.960+0000 7f4a077fe640 1 --2- 172.21.15.40:0/4058892486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a08147f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2309043bbea7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a06ffd640 1 -- 172.21.15.40:0/4058892486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0aa610 con 0x7f4a080aeb80 2024-04-02T16:12:43.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a06ffd640 1 -- 172.21.15.40:0/4058892486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc0af070 con 0x7f4a080aeb80 2024-04-02T16:12:43.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a06ffd640 1 -- 172.21.15.40:0/4058892486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0aa9d0 con 0x7f4a080aeb80 2024-04-02T16:12:43.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/4058892486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 msgr2=0x7f4a08147f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 --2- 172.21.15.40:0/4058892486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a08147f10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc01c040 comp rx=0 tx=0).stop 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/4058892486 shutdown_connections 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 --2- 172.21.15.40:0/4058892486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a08147f10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/4058892486 >> 172.21.15.40:0/4058892486 conn(0x7f4a080b5890 msgr2=0x7f4a080b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/4058892486 shutdown_connections 2024-04-02T16:12:43.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/4058892486 wait complete. 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 Processor -- start 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- start start 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a0806d830 0x7f4a08070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a0806fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a08148450 0x7f4a08070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080b3b20 con 0x7f4a080aeb80 2024-04-02T16:12:43.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080b39a0 con 0x7f4a0806d830 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4a080b3ca0 con 0x7f4a08148450 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a0806fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a0806fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50686/0 (socket says 172.21.15.40:50686) 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 -- 172.21.15.40:0/2802357846 learned_addr learned my addr 172.21.15.40:0/2802357846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a07fff640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a0806d830 0x7f4a08070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0ccbb640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a08148450 0x7f4a08070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 -- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a08148450 msgr2=0x7f4a08070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a08148450 0x7f4a08070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 -- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a0806d830 msgr2=0x7f4a08070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a0806d830 0x7f4a08070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a081444e0 con 0x7f4a080aeb80 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0ccbb640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a08148450 0x7f4a08070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a077fe640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a0806fad0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ac040 tx=0x7f49fc01a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0bd5f0 con 0x7f4a080aeb80 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc0bd7b0 con 0x7f4a080aeb80 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0813dfa0 con 0x7f4a080aeb80 2024-04-02T16:12:43.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0bdba0 con 0x7f4a080aeb80 2024-04-02T16:12:43.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.964+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a08070fb0 con 0x7f4a080aeb80 2024-04-02T16:12:43.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a080685f0 con 0x7f4a080aeb80 2024-04-02T16:12:43.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f49fc0c6440 con 0x7f4a080aeb80 2024-04-02T16:12:43.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49f4068380 0x7f49f406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f49fc120b30 con 0x7f4a080aeb80 2024-04-02T16:12:43.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7f49f406e240 0x7f49f4070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49f4070d10 con 0x7f49f406e240 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f49fc120ee0 con 0x7f4a080aeb80 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a07fff640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49f4068380 0x7f49f406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a0ccbb640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7f49f406e240 0x7f49f4070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a07fff640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49f4068380 0x7f49f406a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000f70 tx=0x7f49f8007c30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.968+0000 7f4a0ccbb640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7f49f406e240 0x7f49f4070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:43.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.972+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== osd.6 v2:172.21.15.204:6803/1944425714 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49f4070d10 con 0x7f49f406e240 2024-04-02T16:12:43.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 --> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a080714f0 con 0x7f49f406e240 2024-04-02T16:12:43.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a04ff9640 1 -- 172.21.15.40:0/2802357846 <== osd.6 v2:172.21.15.204:6803/1944425714 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a080714f0 con 0x7f49f406e240 2024-04-02T16:12:43.987 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:43.987 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:43.987 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7f49f406e240 msgr2=0x7f49f4070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6803/1944425714,v1:172.21.15.204:6807/1944425714] conn(0x7f49f406e240 0x7f49f4070660 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49f4068380 msgr2=0x7f49f406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f49f4068380 0x7f49f406a840 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000f70 tx=0x7f49f8007c30 comp rx=0 tx=0).stop 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 msgr2=0x7f4a0806fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:43.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 --2- 172.21.15.40:0/2802357846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a080aeb80 0x7f4a0806fad0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ac040 tx=0x7f49fc01a880 comp rx=0 tx=0).stop 2024-04-02T16:12:43.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0ccbb640 1 -- 172.21.15.40:0/2802357846 reap_dead start 2024-04-02T16:12:43.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 shutdown_connections 2024-04-02T16:12:43.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 >> 172.21.15.40:0/2802357846 conn(0x7f4a080b5890 msgr2=0x7f4a080b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:43.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 shutdown_connections 2024-04-02T16:12:43.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:43.984+0000 7f4a0e71e640 1 -- 172.21.15.40:0/2802357846 wait complete. 2024-04-02T16:12:44.004 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-02T16:12:44.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 Processor -- start 2024-04-02T16:12:44.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 -- start start 2024-04-02T16:12:44.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06dc80 0x7f80cc0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f80cc073240 0x7f80cc072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc06d030 con 0x7f80cc06cb70 2024-04-02T16:12:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f80cc0b0f80 con 0x7f80cc073240 2024-04-02T16:12:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d3bce640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06d960 con 0x7f80cc06dc80 2024-04-02T16:12:44.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f80cc073240 0x7f80cc072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d1169640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f80cc06cb70 0x7f80cc147c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:57594/0 (socket says 172.21.15.40:57594) 2024-04-02T16:12:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d1169640 1 -- 172.21.15.40:0/671066409 learned_addr learned my addr 172.21.15.40:0/671066409 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 --2- 172.21.15.40:0/671066409 >> v2:172.21.15.204:3300/0 conn(0x7f80cc073240 0x7f80cc072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.40:43986/0 (socket says 172.21.15.40:43986) 2024-04-02T16:12:44.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d196a640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06dc80 0x7f80cc0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 -- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06dc80 msgr2=0x7f80cc0725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06dc80 0x7f80cc0725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 -- 172.21.15.40:0/671066409 >> v1:172.21.15.204:6789/0 conn(0x7f80cc06cb70 legacy=0x7f80cc147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.116+0000 7f80d216b640 1 -- 172.21.15.40:0/671066409 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc06d310 con 0x7f80cc073240 2024-04-02T16:12:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d216b640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc073240 0x7f80cc072ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80c400bfa0 tx=0x7f80c401eae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de9a47734b2c36b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d0968640 1 -- 172.21.15.40:0/671066409 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400b2d0 con 0x7f80cc073240 2024-04-02T16:12:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d0968640 1 -- 172.21.15.40:0/671066409 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c40a77c0 con 0x7f80cc073240 2024-04-02T16:12:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d0968640 1 -- 172.21.15.40:0/671066409 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40a7b90 con 0x7f80cc073240 2024-04-02T16:12:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 -- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc073240 msgr2=0x7f80cc072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc073240 0x7f80cc072ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80c400bfa0 tx=0x7f80c401eae0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 -- 172.21.15.40:0/671066409 shutdown_connections 2024-04-02T16:12:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc073240 0x7f80cc072ae0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 --2- 172.21.15.40:0/671066409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06dc80 0x7f80cc0725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 -- 172.21.15.40:0/671066409 >> 172.21.15.40:0/671066409 conn(0x7f80cc0b7d10 msgr2=0x7f80cc0b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 -- 172.21.15.40:0/671066409 shutdown_connections 2024-04-02T16:12:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.120+0000 7f80d3bce640 1 -- 172.21.15.40:0/671066409 wait complete. 2024-04-02T16:12:44.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 Processor -- start 2024-04-02T16:12:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- start start 2024-04-02T16:12:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06cb70 0x7f80cc13e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc06dc80 0x7f80cc1e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc06c1a0 con 0x7f80cc073240 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc06c020 con 0x7f80cc06dc80 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06c320 con 0x7f80cc06cb70 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50696/0 (socket says 172.21.15.40:50696) 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 -- 172.21.15.40:0/3501806507 learned_addr learned my addr 172.21.15.40:0/3501806507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d1169640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc06dc80 0x7f80cc1e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d196a640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06cb70 0x7f80cc13e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 -- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06cb70 msgr2=0x7f80cc13e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06cb70 0x7f80cc13e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 -- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc06dc80 msgr2=0x7f80cc1e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc06dc80 0x7f80cc1e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 -- 172.21.15.40:0/3501806507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1e3ea0 con 0x7f80cc073240 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d216b640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f80c401c710 tx=0x7f80c401b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40a7df0 con 0x7f80cc073240 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c4009540 con 0x7f80cc073240 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc13fb90 con 0x7f80cc073240 2024-04-02T16:12:44.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4009970 con 0x7f80cc073240 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1e4be0 con 0x7f80cc073240 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc068d00 con 0x7f80cc073240 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f80c40c0020 con 0x7f80cc073240 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.124+0000 7f80c27fc640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 0x7f80b006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.128+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f80c4120fe0 con 0x7f80cc073240 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.128+0000 7f80d196a640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 0x7f80b006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.128+0000 7f80d196a640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 0x7f80b006a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009af0 tx=0x7f80c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.132+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f80c4018760 con 0x7f80cc073240 2024-04-02T16:12:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f80cc0af9d0 con 0x7f80b0068380 2024-04-02T16:12:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80c27fc640 1 -- 172.21.15.40:0/3501806507 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f80cc0af9d0 con 0x7f80b0068380 2024-04-02T16:12:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-04-02T16:12:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 msgr2=0x7f80b006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 0x7f80b006a840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009af0 tx=0x7f80c8008040 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 msgr2=0x7f80cc0b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f80c401c710 tx=0x7f80c401b880 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 shutdown_connections 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f80b0068380 0x7f80b006a840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80cc073240 0x7f80cc0b1210 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80cc06dc80 0x7f80cc1e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 --2- 172.21.15.40:0/3501806507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80cc06cb70 0x7f80cc13e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 >> 172.21.15.40:0/3501806507 conn(0x7f80cc0b7d10 msgr2=0x7f80cc0b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 shutdown_connections 2024-04-02T16:12:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.288+0000 7f80d3bce640 1 -- 172.21.15.40:0/3501806507 wait complete. 2024-04-02T16:12:44.307 INFO:tasks.ceph:Scrubbing osd.7 2024-04-02T16:12:44.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-02T16:12:44.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 Processor -- start 2024-04-02T16:12:44.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 -- start start 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 0x7f29a00ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a00aee40 con 0x7f29a00ae9b0 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f29a00b13e0 con 0x7f29a006d830 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f29a5717640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29a00b3220 con 0x7f29a0148620 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299effd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 0x7f29a00ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.40:32960/0 (socket says 172.21.15.40:32960) 2024-04-02T16:12:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299e7fc640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f29a00ae9b0 0x7f29a0147f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:57606/0 (socket says 172.21.15.40:57606) 2024-04-02T16:12:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 -- 172.21.15.40:0/4209858491 learned_addr learned my addr 172.21.15.40:0/4209858491 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 -- 172.21.15.40:0/4209858491 >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 msgr2=0x7f29a00ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 --2- 172.21.15.40:0/4209858491 >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 0x7f29a00ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299f7fe640 1 -- 172.21.15.40:0/4209858491 >> v1:172.21.15.204:6789/0 conn(0x7f29a00ae9b0 legacy=0x7f29a0147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.420+0000 7f299dffb640 1 -- 172.21.15.40:0/4209858491 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574049359 0 0) 0x7f29a00aee40 con 0x7f29a00ae9b0 2024-04-02T16:12:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299f7fe640 1 -- 172.21.15.40:0/4209858491 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a006dc10 con 0x7f29a0148620 2024-04-02T16:12:44.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299effd640 1 --2- 172.21.15.40:0/4209858491 >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 0x7f29a00ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:12:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299f7fe640 1 --2- 172.21.15.40:0/4209858491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f299801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38dad3e269e3769 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299dffb640 1 -- 172.21.15.40:0/4209858491 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980b0070 con 0x7f29a0148620 2024-04-02T16:12:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299dffb640 1 -- 172.21.15.40:0/4209858491 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29980ab600 con 0x7f29a0148620 2024-04-02T16:12:44.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f299dffb640 1 -- 172.21.15.40:0/4209858491 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980ab9d0 con 0x7f29a0148620 2024-04-02T16:12:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 -- 172.21.15.40:0/4209858491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 msgr2=0x7f29a014ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 --2- 172.21.15.40:0/4209858491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2998009870 tx=0x7f299801c440 comp rx=0 tx=0).stop 2024-04-02T16:12:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 -- 172.21.15.40:0/4209858491 shutdown_connections 2024-04-02T16:12:44.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 --2- 172.21.15.40:0/4209858491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a014ee30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 --2- 172.21.15.40:0/4209858491 >> v2:172.21.15.204:3300/0 conn(0x7f29a006d830 0x7f29a00ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 -- 172.21.15.40:0/4209858491 >> 172.21.15.40:0/4209858491 conn(0x7f29a00b5890 msgr2=0x7f29a00b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 -- 172.21.15.40:0/4209858491 shutdown_connections 2024-04-02T16:12:44.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.424+0000 7f29a5717640 1 -- 172.21.15.40:0/4209858491 wait complete. 2024-04-02T16:12:44.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 Processor -- start 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- start start 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 0x7f29a013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a00ae9b0 0x7f29a01460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a0132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29a0151b90 con 0x7f29a006d830 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29a00b1060 con 0x7f29a00ae9b0 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f29a0151d10 con 0x7f29a0148620 2024-04-02T16:12:44.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 0x7f29a013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 0x7f29a013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50710/0 (socket says 172.21.15.40:50710) 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 -- 172.21.15.40:0/2477061313 learned_addr learned my addr 172.21.15.40:0/2477061313 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299e7fc640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a00ae9b0 0x7f29a01460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299f7fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a0132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 -- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 msgr2=0x7f29a0132fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a0132fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 -- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a00ae9b0 msgr2=0x7f29a01460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a00ae9b0 0x7f29a01460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a0146600 con 0x7f29a006d830 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299f7fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29a0148620 0x7f29a0132fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f299effd640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 0x7f29a013e400 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2990009ec0 tx=0x7f299000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990018020 con 0x7f29a006d830 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990014680 con 0x7f29a006d830 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a0138a90 con 0x7f29a006d830 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990014a50 con 0x7f29a006d830 2024-04-02T16:12:44.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a0141f60 con 0x7f29a006d830 2024-04-02T16:12:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f29a00685f0 con 0x7f29a006d830 2024-04-02T16:12:44.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f299001b020 con 0x7f29a006d830 2024-04-02T16:12:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2980068380 0x7f298006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.428+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f2990087490 con 0x7f29a006d830 2024-04-02T16:12:44.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f29977fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7f298006e240 0x7f2980070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2990016e50 con 0x7f298006e240 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f299008e050 con 0x7f29a006d830 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f299e7fc640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2980068380 0x7f298006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f299f7fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7f298006e240 0x7f2980070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f299f7fe640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7f298006e240 0x7f2980070660 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f299e7fc640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2980068380 0x7f298006a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b0690 tx=0x7f2988009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.432+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== osd.7 v2:172.21.15.204:6802/176733231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2990016e50 con 0x7f298006e240 2024-04-02T16:12:44.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 --> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f29a0072930 con 0x7f298006e240 2024-04-02T16:12:44.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29977fe640 1 -- 172.21.15.40:0/2477061313 <== osd.7 v2:172.21.15.204:6802/176733231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f29a0072930 con 0x7f298006e240 2024-04-02T16:12:44.448 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-02T16:12:44.448 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-02T16:12:44.448 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-02T16:12:44.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7f298006e240 msgr2=0x7f2980070660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6802/176733231,v1:172.21.15.204:6805/176733231] conn(0x7f298006e240 0x7f2980070660 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2980068380 msgr2=0x7f298006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f2980068380 0x7f298006a840 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b0690 tx=0x7f2988009040 comp rx=0 tx=0).stop 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 msgr2=0x7f29a013e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 --2- 172.21.15.40:0/2477061313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29a006d830 0x7f29a013e400 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2990009ec0 tx=0x7f299000c490 comp rx=0 tx=0).stop 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f299f7fe640 1 -- 172.21.15.40:0/2477061313 reap_dead start 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 shutdown_connections 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 >> 172.21.15.40:0/2477061313 conn(0x7f29a00b5890 msgr2=0x7f29a00b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 shutdown_connections 2024-04-02T16:12:44.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.448+0000 7f29a5717640 1 -- 172.21.15.40:0/2477061313 wait complete. 2024-04-02T16:12:44.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-02T16:12:44.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 Processor -- start 2024-04-02T16:12:44.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 -- start start 2024-04-02T16:12:44.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1aa40b1c10 0x7f1aa40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40b25b0 0x7f1aa41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa40b0a10 con 0x7f1aa40ae080 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa40b3210 con 0x7f1aa40b1c10 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aaa609640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa406db10 con 0x7f1aa40b25b0 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1aa40b1c10 0x7f1aa40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1aa40b1c10 0x7f1aa40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50712/0 (socket says 172.21.15.40:50712) 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 -- 172.21.15.40:0/1000608448 learned_addr learned my addr 172.21.15.40:0/1000608448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa8ba6640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40b25b0 0x7f1aa41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 -- 172.21.15.40:0/1000608448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40b25b0 msgr2=0x7f1aa41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:44.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40b25b0 0x7f1aa41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 -- 172.21.15.40:0/1000608448 >> v1:172.21.15.204:6789/0 conn(0x7f1aa40ae080 legacy=0x7f1aa40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.576+0000 7f1aa37fe640 1 -- 172.21.15.40:0/1000608448 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa40ae460 con 0x7f1aa40b1c10 2024-04-02T16:12:44.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aa37fe640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b1c10 0x7f1aa40b2070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009870 tx=0x7f1a8c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f091e40f31e3623c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aa2ffd640 1 -- 172.21.15.40:0/1000608448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0b2070 con 0x7f1aa40b1c10 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aa2ffd640 1 -- 172.21.15.40:0/1000608448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c0a9070 con 0x7f1aa40b1c10 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aa2ffd640 1 -- 172.21.15.40:0/1000608448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0ad880 con 0x7f1aa40b1c10 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- 172.21.15.40:0/1000608448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b1c10 msgr2=0x7f1aa40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b1c10 0x7f1aa40b2070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c009870 tx=0x7f1a8c01c560 comp rx=0 tx=0).stop 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- 172.21.15.40:0/1000608448 shutdown_connections 2024-04-02T16:12:44.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40b25b0 0x7f1aa41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 --2- 172.21.15.40:0/1000608448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b1c10 0x7f1aa40b2070 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- 172.21.15.40:0/1000608448 >> 172.21.15.40:0/1000608448 conn(0x7f1aa40b8fd0 msgr2=0x7f1aa40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- 172.21.15.40:0/1000608448 shutdown_connections 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- 172.21.15.40:0/1000608448 wait complete. 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 Processor -- start 2024-04-02T16:12:44.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.580+0000 7f1aaa609640 1 -- start start 2024-04-02T16:12:44.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40ae080 0x7f1aa414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa40b1c10 0x7f1aa414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa40af2f0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa40ae760 con 0x7f1aa40b1c10 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa40aeeb0 con 0x7f1aa40ae080 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50722/0 (socket says 172.21.15.40:50722) 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 -- 172.21.15.40:0/2547042165 learned_addr learned my addr 172.21.15.40:0/2547042165 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa37fe640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa40b1c10 0x7f1aa414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa3fff640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40ae080 0x7f1aa414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 -- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40ae080 msgr2=0x7f1aa414f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40ae080 0x7f1aa414f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 -- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa40b1c10 msgr2=0x7f1aa414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa40b1c10 0x7f1aa414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 -- 172.21.15.40:0/2547042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa41e00f0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa8ba6640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d560 tx=0x7f1a9400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940125f0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a94019070 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa41515c0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940129f0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41512e0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa40685f0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f1a94012b90 con 0x7f1aa40b25b0 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 0x7f1a8406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f1a940869e0 con 0x7f1aa40b25b0 2024-04-02T16:12:44.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.584+0000 7f1aa3fff640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 0x7f1a8406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.588+0000 7f1aa3fff640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 0x7f1a8406a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c001d50 tx=0x7f1a9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.592+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f1a94057330 con 0x7f1aa40b25b0 2024-04-02T16:12:44.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.752+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f1aa4151a50 con 0x7f1a84068380 2024-04-02T16:12:44.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.752+0000 7f1aa0ff9640 1 -- 172.21.15.40:0/2547042165 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1aa4151a50 con 0x7f1a84068380 2024-04-02T16:12:44.754 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.752+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 msgr2=0x7f1a8406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 0x7f1a8406a840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c001d50 tx=0x7f1a9c009040 comp rx=0 tx=0).stop 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 msgr2=0x7f1aa41e05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d560 tx=0x7f1a9400da20 comp rx=0 tx=0).stop 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 shutdown_connections 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f1a84068380 0x7f1a8406a840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa40b25b0 0x7f1aa41e05d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aa40b1c10 0x7f1aa414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 --2- 172.21.15.40:0/2547042165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aa40ae080 0x7f1aa414f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 >> 172.21.15.40:0/2547042165 conn(0x7f1aa40b8fd0 msgr2=0x7f1aa40b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 shutdown_connections 2024-04-02T16:12:44.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.756+0000 7f1aaa609640 1 -- 172.21.15.40:0/2547042165 wait complete. 2024-04-02T16:12:44.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:12:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.896+0000 7fd7bb63a640 1 Processor -- start 2024-04-02T16:12:44.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.896+0000 7fd7bb63a640 1 -- start start 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7b406c8a0 0x7fd7b40af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b40b3800 0x7fd7b40b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b40b4640 con 0x7fd7b40b41d0 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b40b2b50 con 0x7fd7b40b3800 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b40b0300 con 0x7fd7b406c8a0 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b40b3800 0x7fd7b40b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b9bd7640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7b40b41d0 0x7fd7b4147c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34158/0 (socket says 172.21.15.40:34158) 2024-04-02T16:12:44.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b9bd7640 1 -- 172.21.15.40:0/2095411060 learned_addr learned my addr 172.21.15.40:0/2095411060 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd7b40b3800 0x7fd7b40b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50724/0 (socket says 172.21.15.40:50724) 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b93d6640 1 --2- 172.21.15.40:0/2095411060 >> v2:172.21.15.204:3300/0 conn(0x7fd7b406c8a0 0x7fd7b40af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/2095411060 >> v2:172.21.15.204:3300/0 conn(0x7fd7b406c8a0 msgr2=0x7fd7b40af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 --2- 172.21.15.40:0/2095411060 >> v2:172.21.15.204:3300/0 conn(0x7fd7b406c8a0 0x7fd7b40af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/2095411060 >> v1:172.21.15.40:6789/0 conn(0x7fd7b40b41d0 legacy=0x7fd7b4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/2095411060 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b40ae830 con 0x7fd7b40b3800 2024-04-02T16:12:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7abfff640 1 -- 172.21.15.40:0/2095411060 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051864981 0 0) 0x7fd7b40b4640 con 0x7fd7b40b41d0 2024-04-02T16:12:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7b8bd5640 1 --2- 172.21.15.40:0/2095411060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b40b3c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406e070 tx=0x7fd7a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9c2cf4c43d91a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7abfff640 1 -- 172.21.15.40:0/2095411060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b0070 con 0x7fd7b40b3800 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7abfff640 1 -- 172.21.15.40:0/2095411060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a40ab600 con 0x7fd7b40b3800 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7abfff640 1 -- 172.21.15.40:0/2095411060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40ab9d0 con 0x7fd7b40b3800 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- 172.21.15.40:0/2095411060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 msgr2=0x7fd7b40b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/2095411060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b40b3c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7b406e070 tx=0x7fd7a401c3f0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- 172.21.15.40:0/2095411060 shutdown_connections 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/2095411060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b40b3c90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/2095411060 >> v2:172.21.15.204:3300/0 conn(0x7fd7b406c8a0 0x7fd7b40af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- 172.21.15.40:0/2095411060 >> 172.21.15.40:0/2095411060 conn(0x7fd7b40b8fd0 msgr2=0x7fd7b40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- 172.21.15.40:0/2095411060 shutdown_connections 2024-04-02T16:12:44.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.900+0000 7fd7bb63a640 1 -- 172.21.15.40:0/2095411060 wait complete. 2024-04-02T16:12:44.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 Processor -- start 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- start start 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7b406c8a0 0x7fd7b4146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 0x7fd7b41510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40af690 con 0x7fd7b40b3800 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b40b0540 con 0x7fd7b40b41d0 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b40b3130 con 0x7fd7b406c8a0 2024-04-02T16:12:44.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50740/0 (socket says 172.21.15.40:50740) 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/3984699150 learned_addr learned my addr 172.21.15.40:0/3984699150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b9bd7640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 0x7fd7b41510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b93d6640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7b406c8a0 0x7fd7b4146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7b406c8a0 msgr2=0x7fd7b4146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7b406c8a0 0x7fd7b4146ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 msgr2=0x7fd7b41510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 0x7fd7b41510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 -- 172.21.15.40:0/3984699150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b409c930 con 0x7fd7b40b3800 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b9bd7640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 0x7fd7b41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7b8bd5640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4019000 tx=0x7fd7a4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b8020 con 0x7fd7b40b3800 2024-04-02T16:12:44.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a40b4610 con 0x7fd7b40b3800 2024-04-02T16:12:44.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b409cc10 con 0x7fd7b40b3800 2024-04-02T16:12:44.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40b0040 con 0x7fd7b40b3800 2024-04-02T16:12:44.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b409ba20 con 0x7fd7b40b3800 2024-04-02T16:12:44.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.904+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b41525f0 con 0x7fd7b40b3800 2024-04-02T16:12:44.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.908+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd7a40bb020 con 0x7fd7b40b3800 2024-04-02T16:12:44.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.908+0000 7fd7a9ffb640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 0x7fd79806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:12:44.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.908+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd7a4120640 con 0x7fd7b40b3800 2024-04-02T16:12:44.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.908+0000 7fd7b93d6640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 0x7fd79806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:12:44.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.908+0000 7fd7b93d6640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 0x7fd79806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00096f0 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:12:44.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:44.912+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd7a40c2030 con 0x7fd7b40b3800 2024-04-02T16:12:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.072+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7b409c650 con 0x7fd798068380 2024-04-02T16:12:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.072+0000 7fd7a9ffb640 1 -- 172.21.15.40:0/3984699150 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7fd7b000e4c0 con 0x7fd798068380 2024-04-02T16:12:45.075 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:12:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:12:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 msgr2=0x7fd79806a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 0x7fd79806a840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00096f0 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2024-04-02T16:12:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 msgr2=0x7fd7b4152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:12:45.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4019000 tx=0x7fd7a4019190 comp rx=0 tx=0).stop 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 shutdown_connections 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd798068380 0x7fd79806a840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7b40b41d0 0x7fd7b41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7b40b3800 0x7fd7b4152ac0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 --2- 172.21.15.40:0/3984699150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7b406c8a0 0x7fd7b4146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 >> 172.21.15.40:0/3984699150 conn(0x7fd7b40b8fd0 msgr2=0x7fd7b40b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 shutdown_connections 2024-04-02T16:12:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:12:45.076+0000 7fd7bb63a640 1 -- 172.21.15.40:0/3984699150 wait complete. 2024-04-02T16:12:45.094 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-02T16:12:43.872754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224388,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750332,"statfs":{"total":773094113280,"available":772864339968,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002665"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-02T16:12:37.580125+0000","last_change":"2024-04-02T16:11:22.027466+0000","last_active":"2024-04-02T16:12:37.580125+0000","last_peered":"2024-04-02T16:12:37.580125+0000","last_clean":"2024-04-02T16:12:37.580125+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:37.580125+0000","last_undegraded":"2024-04-02T16:12:37.580125+0000","last_fullsized":"2024-04-02T16:12:37.580125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T16:11:20.023957+0000","last_clean_scrub_stamp":"2024-04-02T16:11:20.023957+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:12:00.539686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:12:45.096 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-02T16:11:20.023957+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=16, tm_min=11, tm_sec=20, tm_wday=1, tm_yday=93, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=2, tm_hour=16, tm_min=12, tm_sec=39, tm_wday=1, tm_yday=93, tm_isdst=0) 2024-04-02T16:12:45.096 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-02T16:13:02.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:02.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:05.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T16:13:05.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 Processor -- start 2024-04-02T16:13:05.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 -- start start 2024-04-02T16:13:05.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b7470 0x7fd994141480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd9940b5940 0x7fd9941419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd99406d490 con 0x7fd9940b80a0 2024-04-02T16:13:05.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd9940b8480 con 0x7fd9940b5940 2024-04-02T16:13:05.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd99a668640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd9940b5dd0 con 0x7fd9940b7470 2024-04-02T16:13:05.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd998c05640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd9940b5940 0x7fd9941419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b7470 0x7fd994141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd993fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd9940b80a0 0x7fd99406da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.40:57136/0 (socket says 172.21.15.40:57136) 2024-04-02T16:13:05.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd993fff640 1 -- 172.21.15.40:0/1970941408 learned_addr learned my addr 172.21.15.40:0/1970941408 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:13:05.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd998c05640 1 -- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b7470 msgr2=0x7fd994141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:13:05.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd998c05640 1 --2- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b7470 0x7fd994141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd998c05640 1 -- 172.21.15.40:0/1970941408 >> v1:172.21.15.204:6789/0 conn(0x7fd9940b80a0 legacy=0x7fd99406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.232+0000 7fd998c05640 1 -- 172.21.15.40:0/1970941408 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9940b60b0 con 0x7fd9940b5940 2024-04-02T16:13:05.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd998c05640 1 --2- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b5940 0x7fd9941419c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009d90 tx=0x7fd98c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f324bcf615544ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd992ffd640 1 -- 172.21.15.40:0/1970941408 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0b2070 con 0x7fd9940b5940 2024-04-02T16:13:05.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd992ffd640 1 -- 172.21.15.40:0/1970941408 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c0a5850 con 0x7fd9940b5940 2024-04-02T16:13:05.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd992ffd640 1 -- 172.21.15.40:0/1970941408 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0a5c20 con 0x7fd9940b5940 2024-04-02T16:13:05.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b5940 msgr2=0x7fd9941419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b5940 0x7fd9941419c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009d90 tx=0x7fd98c01c8e0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- 172.21.15.40:0/1970941408 shutdown_connections 2024-04-02T16:13:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b5940 0x7fd9941419c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- 172.21.15.40:0/1970941408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b7470 0x7fd994141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- 172.21.15.40:0/1970941408 >> 172.21.15.40:0/1970941408 conn(0x7fd99406b980 msgr2=0x7fd99406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- 172.21.15.40:0/1970941408 shutdown_connections 2024-04-02T16:13:05.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- 172.21.15.40:0/1970941408 wait complete. 2024-04-02T16:13:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 Processor -- start 2024-04-02T16:13:05.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 -- start start 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 0x7fd99413c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 0x7fd9941e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.236+0000 7fd99a668640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd994154a80 con 0x7fd994139c00 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd994154900 con 0x7fd9940b7470 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd994154c00 con 0x7fd9940b80a0 2024-04-02T16:13:05.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49706/0 (socket says 172.21.15.40:49706) 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 -- 172.21.15.40:0/184126340 learned_addr learned my addr 172.21.15.40:0/184126340 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd9937fe640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 0x7fd9941e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd993fff640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 0x7fd99413c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 -- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 msgr2=0x7fd9941e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 0x7fd9941e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 -- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 msgr2=0x7fd99413c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 0x7fd99413c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 -- 172.21.15.40:0/184126340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd994139a10 con 0x7fd994139c00 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd993fff640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 0x7fd99413c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd9937fe640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 0x7fd9941e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd998c05640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0b0040 tx=0x7fd98c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0b2050 con 0x7fd994139c00 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c0be070 con 0x7fd994139c00 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd994139740 con 0x7fd994139c00 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0ada90 con 0x7fd994139c00 2024-04-02T16:13:05.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99413b0b0 con 0x7fd994139c00 2024-04-02T16:13:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9940685f0 con 0x7fd994139c00 2024-04-02T16:13:05.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fd98c0a56d0 con 0x7fd994139c00 2024-04-02T16:13:05.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 0x7fd98006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd98c121840 con 0x7fd994139c00 2024-04-02T16:13:05.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.240+0000 7fd993fff640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 0x7fd98006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.244+0000 7fd993fff640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 0x7fd98006a840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd994145740 tx=0x7fd984009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.244+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fd98c016560 con 0x7fd994139c00 2024-04-02T16:13:05.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.404+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 --> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9940b3c00 con 0x7fd980068380 2024-04-02T16:13:05.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.404+0000 7fd990ff9640 1 -- 172.21.15.40:0/184126340 <== mgr.4105 v2:172.21.15.204:6832/1155217760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7fd984001c00 con 0x7fd980068380 2024-04-02T16:13:05.407 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:13:05.407 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 msgr2=0x7fd98006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 0x7fd98006a840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd994145740 tx=0x7fd984009040 comp rx=0 tx=0).stop 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 msgr2=0x7fd9941371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0b0040 tx=0x7fd98c019680 comp rx=0 tx=0).stop 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 shutdown_connections 2024-04-02T16:13:05.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9940b80a0 0x7fd9941e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fd980068380 0x7fd98006a840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd994139c00 0x7fd9941371b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 --2- 172.21.15.40:0/184126340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9940b7470 0x7fd99413c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 >> 172.21.15.40:0/184126340 conn(0x7fd99406b980 msgr2=0x7fd994147840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:05.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 shutdown_connections 2024-04-02T16:13:05.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.408+0000 7fd99a668640 1 -- 172.21.15.40:0/184126340 wait complete. 2024-04-02T16:13:05.426 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-02T16:13:03.878352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224376,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750344,"statfs":{"total":773094113280,"available":772864352256,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003479"},"pg_stats":[{"pgid":"1.0","version":"32'119","reported_seq":205,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-02T16:12:45.227303+0000","last_change":"2024-04-02T16:12:45.227395+0000","last_active":"2024-04-02T16:12:45.227303+0000","last_peered":"2024-04-02T16:12:45.227303+0000","last_clean":"2024-04-02T16:12:45.227303+0000","last_became_active":"2024-04-02T16:11:22.021648+0000","last_became_peered":"2024-04-02T16:11:22.021648+0000","last_unstale":"2024-04-02T16:12:45.227303+0000","last_undegraded":"2024-04-02T16:12:45.227303+0000","last_fullsized":"2024-04-02T16:12:45.227303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-02T16:12:45.227116+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-02T16:12:45.227116+0000","last_clean_scrub_stamp":"2024-04-02T16:12:45.227116+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:12:43.038661+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T16:13:05.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-02T16:13:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 Processor -- start 2024-04-02T16:13:05.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 -- start start 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcd40b23d0 0x7fdcd40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd40b19e0 con 0x7fdcd4073230 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd40b05c0 con 0x7fdcd4148e50 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcdabbc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd40b08a0 con 0x7fdcd40b23d0 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcd8958640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcd40b23d0 0x7fdcd40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcd8958640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdcd40b23d0 0x7fdcd40b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49718/0 (socket says 172.21.15.40:49718) 2024-04-02T16:13:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcd8958640 1 -- 172.21.15.40:0/212429218 learned_addr learned my addr 172.21.15.40:0/212429218 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:13:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.540+0000 7fdcd3fff640 1 --1- 172.21.15.40:0/212429218 >> v1:172.21.15.40:6789/0 conn(0x7fdcd4073230 0x7fdcd409daa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42386/0 (socket says 172.21.15.40:42386) 2024-04-02T16:13:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd8958640 1 -- 172.21.15.40:0/212429218 >> v1:172.21.15.204:6789/0 conn(0x7fdcd4148e50 legacy=0x7fdcd4149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:13:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd8958640 1 -- 172.21.15.40:0/212429218 >> v1:172.21.15.40:6789/0 conn(0x7fdcd4073230 legacy=0x7fdcd409daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd8958640 1 -- 172.21.15.40:0/212429218 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd40b0b80 con 0x7fdcd40b23d0 2024-04-02T16:13:05.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd8958640 1 --2- 172.21.15.40:0/212429218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd40b23d0 0x7fdcd40b27b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009870 tx=0x7fdccc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a6002a80141e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd37fe640 1 -- 172.21.15.40:0/212429218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0b1070 con 0x7fdcd40b23d0 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd37fe640 1 -- 172.21.15.40:0/212429218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdccc0ac410 con 0x7fdcd40b23d0 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcd37fe640 1 -- 172.21.15.40:0/212429218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0ac7e0 con 0x7fdcd40b23d0 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 -- 172.21.15.40:0/212429218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd40b23d0 msgr2=0x7fdcd40b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/212429218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd40b23d0 0x7fdcd40b27b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009870 tx=0x7fdccc01c490 comp rx=0 tx=0).stop 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 -- 172.21.15.40:0/212429218 shutdown_connections 2024-04-02T16:13:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/212429218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd40b23d0 0x7fdcd40b27b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 -- 172.21.15.40:0/212429218 >> 172.21.15.40:0/212429218 conn(0x7fdcd40b8fd0 msgr2=0x7fdcd40b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:05.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 -- 172.21.15.40:0/212429218 shutdown_connections 2024-04-02T16:13:05.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.544+0000 7fdcdabbc640 1 -- 172.21.15.40:0/212429218 wait complete. 2024-04-02T16:13:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 Processor -- start 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- start start 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd4073230 0x7fdcd40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 0x7fdcd41394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4153a30 con 0x7fdcd4148e50 2024-04-02T16:13:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41538b0 con 0x7fdcd40b23d0 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd4153bb0 con 0x7fdcd4073230 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49724/0 (socket says 172.21.15.40:49724) 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 -- 172.21.15.40:0/428032820 learned_addr learned my addr 172.21.15.40:0/428032820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd3fff640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 0x7fdcd41394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd8958640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd4073230 0x7fdcd40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 -- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd4073230 msgr2=0x7fdcd40af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd4073230 0x7fdcd40af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 -- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 msgr2=0x7fdcd41394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 0x7fdcd41394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 -- 172.21.15.40:0/428032820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd413dae0 con 0x7fdcd4148e50 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd3fff640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 0x7fdcd41394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd9159640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400b440 tx=0x7fdcc400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4015020 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc4011850 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4011c40 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd413dd80 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd413f0f0 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd413f580 con 0x7fdcd4148e50 2024-04-02T16:13:05.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7fdcc4018020 con 0x7fdcd4148e50 2024-04-02T16:13:05.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.548+0000 7fdcd17fa640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 0x7fdcc006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.552+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdcc40861f0 con 0x7fdcd4148e50 2024-04-02T16:13:05.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.552+0000 7fdcd8958640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 0x7fdcc006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.552+0000 7fdcd8958640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 0x7fdcc006a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ae000 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.556+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7fdcc401f030 con 0x7fdcd4148e50 2024-04-02T16:13:05.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.716+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdcd40685f0 con 0x7fdcd4148e50 2024-04-02T16:13:05.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.720+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fdcc4018310 con 0x7fdcd4148e50 2024-04-02T16:13:05.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcd17fa640 1 -- 172.21.15.40:0/428032820 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fdcc40563a0 con 0x7fdcd4148e50 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 msgr2=0x7fdcc006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 0x7fdcc006a840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ae000 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 msgr2=0x7fdcd40aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400b440 tx=0x7fdcc400b900 comp rx=0 tx=0).stop 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 shutdown_connections 2024-04-02T16:13:05.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7fdcc0068380 0x7fdcc006a840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcd4148e50 0x7fdcd40aecd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcd40b23d0 0x7fdcd41394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 --2- 172.21.15.40:0/428032820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcd4073230 0x7fdcd40af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 >> 172.21.15.40:0/428032820 conn(0x7fdcd40b8fd0 msgr2=0x7fdcd406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 shutdown_connections 2024-04-02T16:13:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.724+0000 7fdcdabbc640 1 -- 172.21.15.40:0/428032820 wait complete. 2024-04-02T16:13:05.742 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T16:13:05.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T16:13:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.864+0000 7f52ee288640 1 Processor -- start 2024-04-02T16:13:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.864+0000 7f52ee288640 1 -- start start 2024-04-02T16:13:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ee288640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e80bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ee288640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 0x7f52e8148e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ee288640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e80b52e0 con 0x7f52e81493c0 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ee288640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f52e80b6f00 con 0x7f52e80b6170 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ee288640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80b71a0 con 0x7f52e80b4f00 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ec825640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52e81493c0 0x7f52e8071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42396/0 (socket says 172.21.15.40:42396) 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52ec825640 1 -- 172.21.15.40:0/924786412 learned_addr learned my addr 172.21.15.40:0/924786412 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 --2- 172.21.15.40:0/924786412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e80bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e77fe640 1 --2- 172.21.15.40:0/924786412 >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 0x7f52e8148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337457149 0 0) 0x7f52e00014a0 con 0x7f52e81493c0 2024-04-02T16:13:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52d0003530 con 0x7f52e81493c0 2024-04-02T16:13:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2875175638 0 0) 0x7f52d0003530 con 0x7f52e81493c0 2024-04-02T16:13:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 -- 172.21.15.40:0/924786412 >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 msgr2=0x7f52e8148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T16:13:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 --2- 172.21.15.40:0/924786412 >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 0x7f52e8148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 -- 172.21.15.40:0/924786412 >> v1:172.21.15.40:6789/0 conn(0x7f52e81493c0 legacy=0x7f52e8071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 -- 172.21.15.40:0/924786412 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e81497d0 con 0x7f52e80b4f00 2024-04-02T16:13:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e77fe640 1 --2- 172.21.15.40:0/924786412 >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 0x7f52e8148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T16:13:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.868+0000 7f52e7fff640 1 --2- 172.21.15.40:0/924786412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e80bb300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097c0 tx=0x7f52d801c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabb3b87a9994e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80b1070 con 0x7f52e80b4f00 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f52d80a4cb0 con 0x7f52e80b4f00 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52e6ffd640 1 -- 172.21.15.40:0/924786412 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0001bd0 con 0x7f52e80b4f00 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- 172.21.15.40:0/924786412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 msgr2=0x7f52e80bb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- 172.21.15.40:0/924786412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e80bb300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097c0 tx=0x7f52d801c2a0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- 172.21.15.40:0/924786412 shutdown_connections 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- 172.21.15.40:0/924786412 >> v2:172.21.15.204:3300/0 conn(0x7f52e80b6170 0x7f52e8148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- 172.21.15.40:0/924786412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e80bb300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- 172.21.15.40:0/924786412 >> 172.21.15.40:0/924786412 conn(0x7f52e802c5d0 msgr2=0x7f52e802c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- 172.21.15.40:0/924786412 shutdown_connections 2024-04-02T16:13:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- 172.21.15.40:0/924786412 wait complete. 2024-04-02T16:13:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 Processor -- start 2024-04-02T16:13:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- start start 2024-04-02T16:13:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e8145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e80b6170 0x7f52e8141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b5950 con 0x7f52e81493c0 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b57d0 con 0x7f52e80b6170 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ee288640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f52e8152e60 con 0x7f52e80b4f00 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49726/0 (socket says 172.21.15.40:49726) 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 -- 172.21.15.40:0/2721327377 learned_addr learned my addr 172.21.15.40:0/2721327377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52e7fff640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e8145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52e77fe640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e80b6170 0x7f52e8141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 -- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 msgr2=0x7f52e8145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e8145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 -- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e80b6170 msgr2=0x7f52e8141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e80b6170 0x7f52e8141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 -- 172.21.15.40:0/2721327377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e813a0a0 con 0x7f52e81493c0 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.872+0000 7f52ec825640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52e0001450 tx=0x7f52e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0012400 con 0x7f52e81493c0 2024-04-02T16:13:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f52e00125c0 con 0x7f52e81493c0 2024-04-02T16:13:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e8146010 con 0x7f52e81493c0 2024-04-02T16:13:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e00129d0 con 0x7f52e81493c0 2024-04-02T16:13:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e81414f0 con 0x7f52e81493c0 2024-04-02T16:13:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8068d00 con 0x7f52e81493c0 2024-04-02T16:13:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) ==== 87987+0+0 (secure 0 0 0) 0x7f52e0012b70 con 0x7f52e81493c0 2024-04-02T16:13:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 0x7f52d406a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T16:13:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f52e0087980 con 0x7f52e81493c0 2024-04-02T16:13:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e7fff640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 0x7f52d406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T16:13:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.876+0000 7f52e7fff640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 0x7f52d406a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52d80ae000 tx=0x7f52d80a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T16:13:05.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:05.880+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179602 (secure 0 0 0) 0x7f52e80b52e0 con 0x7f52e81493c0 2024-04-02T16:13:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52e80bab60 con 0x7f52e81493c0 2024-04-02T16:13:06.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52e4ff9640 1 -- 172.21.15.40:0/2721327377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+2222 (secure 0 0 0) 0x7f52e00583b0 con 0x7f52e81493c0 2024-04-02T16:13:06.081 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-02T16:13:06.082 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2595009767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2595009767},{"type":"v1","addr":"172.21.15.204:6835","nonce":2595009767}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1064349247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1064349247},{"type":"v1","addr":"172.21.15.40:6833","nonce":1064349247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-04-02T16:13:06.082 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2024-04-02T16:13:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 msgr2=0x7f52d406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 0x7f52d406a840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52d80ae000 tx=0x7f52d80a6070 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 msgr2=0x7f52e813ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52e0001450 tx=0x7f52e000a430 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 shutdown_connections 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:6832/1155217760,v1:172.21.15.204:6833/1155217760] conn(0x7f52d4068380 0x7f52d406a840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52e81493c0 0x7f52e813ff10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e80b6170 0x7f52e8141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 --2- 172.21.15.40:0/2721327377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e80b4f00 0x7f52e8145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T16:13:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 >> 172.21.15.40:0/2721327377 conn(0x7f52e802c5d0 msgr2=0x7f52e80b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T16:13:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 shutdown_connections 2024-04-02T16:13:06.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-02T16:13:06.080+0000 7f52ee288640 1 -- 172.21.15.40:0/2721327377 wait complete. 2024-04-02T16:13:06.100 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-02T16:13:06.100 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-02T16:13:06.101 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-02T16:13:06.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:06.112 INFO:tasks.ceph.mds.a:Stopped 2024-04-02T16:13:06.112 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-02T16:13:06.112 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:06.128 INFO:tasks.ceph.mds.b:Stopped 2024-04-02T16:13:06.129 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-02T16:13:06.129 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-02T16:13:06.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:08.963 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-02T16:13:12.231 INFO:tasks.ceph.osd.0:Stopped 2024-04-02T16:13:12.232 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-02T16:13:12.232 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:12.306 INFO:tasks.ceph.osd.1:Stopped 2024-04-02T16:13:12.307 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-02T16:13:12.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:12.394 INFO:tasks.ceph.osd.2:Stopped 2024-04-02T16:13:12.395 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-02T16:13:12.395 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:12.488 INFO:tasks.ceph.osd.3:Stopped 2024-04-02T16:13:12.488 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-02T16:13:12.488 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:12.564 INFO:tasks.ceph.osd.4:Stopped 2024-04-02T16:13:12.565 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-02T16:13:12.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:14.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-02T16:13:14.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-02T16:13:18.667 INFO:tasks.ceph.osd.5:Stopped 2024-04-02T16:13:18.667 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-02T16:13:18.668 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.731 INFO:tasks.ceph.osd.6:Stopped 2024-04-02T16:13:18.732 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-02T16:13:18.732 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.827 INFO:tasks.ceph.osd.7:Stopped 2024-04-02T16:13:18.827 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-02T16:13:18.828 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-02T16:13:18.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.865 INFO:tasks.ceph.mgr.y:Stopped 2024-04-02T16:13:18.865 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-02T16:13:18.865 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.894 INFO:tasks.ceph.mgr.x:Stopped 2024-04-02T16:13:18.894 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-02T16:13:18.894 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-02T16:13:18.895 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.925 INFO:tasks.ceph.mon.a:Stopped 2024-04-02T16:13:18.925 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-02T16:13:18.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:18.947 INFO:tasks.ceph.mon.b:Stopped 2024-04-02T16:13:18.947 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-02T16:13:18.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T16:13:19.009 INFO:tasks.ceph.mon.c:Stopped 2024-04-02T16:13:19.009 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-02T16:13:19.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-02T16:13:19.028 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:13:19.028 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-02T16:13:19.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:13:19.230 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-02T16:13:19.418 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:13:19.418 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-02T16:13:19.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-02T16:13:19.703 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:13:19.704 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-02T16:13:19.868 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:13:19.869 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-02T16:13:20.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:13:20.143 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-02T16:13:20.317 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:13:20.317 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-02T16:13:20.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:13:20.478 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-02T16:13:20.651 INFO:tasks.ceph:Archiving mon data... 2024-04-02T16:13:20.653 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/data/mon.a.tgz 2024-04-02T16:13:20.653 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-04-02T16:13:20.659 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.aA4N4t9UGU 2024-04-02T16:13:20.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.aA4N4t9UGU 2024-04-02T16:13:20.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.aA4N4t9UGU 2024-04-02T16:13:20.831 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.aA4N4t9UGU is 274KB 2024-04-02T16:13:20.893 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.aA4N4t9UGU 2024-04-02T16:13:20.898 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/data/mon.b.tgz 2024-04-02T16:13:20.898 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-02T16:13:20.904 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.Xk8IZa7rjq 2024-04-02T16:13:20.905 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Xk8IZa7rjq 2024-04-02T16:13:21.036 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.Xk8IZa7rjq 2024-04-02T16:13:21.104 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.Xk8IZa7rjq is 371KB 2024-04-02T16:13:21.166 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.Xk8IZa7rjq 2024-04-02T16:13:21.171 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/data/mon.c.tgz 2024-04-02T16:13:21.171 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-04-02T16:13:21.219 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.fj57kZbRNU 2024-04-02T16:13:21.220 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fj57kZbRNU 2024-04-02T16:13:21.346 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.fj57kZbRNU 2024-04-02T16:13:21.412 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.fj57kZbRNU is 371KB 2024-04-02T16:13:21.473 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.fj57kZbRNU 2024-04-02T16:13:21.479 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-02T16:13:21.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-02T16:13:21.481 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-02T16:13:21.541 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-02T16:13:21.584 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-02T16:13:21.584 INFO:tasks.ceph:Archiving crash dumps... 2024-04-02T16:13:21.586 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi040/crash 2024-04-02T16:13:21.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-02T16:13:21.601 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi204/crash 2024-04-02T16:13:21.602 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-02T16:13:21.640 INFO:tasks.ceph:Shutting down logrotate 2024-04-02T16:13:21.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:21.649 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:21.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:21.753 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-02T16:13:21.767 INFO:tasks.ceph:Compressing logs... 2024-04-02T16:13:21.767 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T16:13:21.797 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T16:13:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35536.log 2024-04-02T16:13:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32840.log 2024-04-02T16:13:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35536.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35536.log.gzgzip 2024-04-02T16:13:21.816 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37743.log 2024-04-02T16:13:21.817 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34878.log 2024-04-02T16:13:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.32840.log: /var/log/ceph/ceph-client.admin.37743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35112.log 2024-04-02T16:13:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32840.log.gz 2024-04-02T16:13:21.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34878.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.37743.log.gz -5 2024-04-02T16:13:21.818 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-04-02T16:13:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35112.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34878.log.gz 2024-04-02T16:13:21.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-02T16:13:21.823 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35112.log.gz 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.34148.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37639.log 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36061.log 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33526.log 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37639.log.gz 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35060.log 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36061.log.gz 2024-04-02T16:13:21.824 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37952.log 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33526.log: /var/log/ceph/ceph-client.admin.35060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37138.log 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33526.log.gz 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37952.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35060.log.gz 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35778.log 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37952.log.gz 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35830.log 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37138.log: /var/log/ceph/ceph-client.admin.35778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35138.log 2024-04-02T16:13:21.825 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37138.log.gz 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35830.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35778.log.gz 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.35830.log.gz --verbose 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.34486.log 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35138.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33394.log 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34486.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35138.log.gz 2024-04-02T16:13:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34330.log 2024-04-02T16:13:21.844 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27666.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27666.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27666.log.gz 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27714.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27714.log.gz 2024-04-02T16:13:21.845 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27618.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27618.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27618.log.gz 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27762.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-02T16:13:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27738.log 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27762.log.gz 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27690.log 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27738.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27690.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.27738.log.gz 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27690.log.gz 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-02T16:13:21.847 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-02T16:13:21.848 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-02T16:13:21.848 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27594.log 2024-04-02T16:13:21.858 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.27594.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27594.log.gz 2024-04-02T16:13:21.858 INFO:teuthology.orchestra.run.smithi204.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-02T16:13:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34486.log.gz 2024-04-02T16:13:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-04-02T16:13:21.867 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33394.log.gz 2024-04-02T16:13:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-04-02T16:13:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36035.log 88.5% 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34330.log.gz 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36728.log: /var/log/ceph/ceph-client.admin.34904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37978.log 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36035.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36035.log.gz 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35242.log 2024-04-02T16:13:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37978.log: /var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35268.log 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37978.log.gz 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36977.log 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35242.log.gz/var/log/ceph/ceph-client.admin.35268.log: 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34304.log 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36977.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35268.log.gz 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-04-02T16:13:21.869 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36977.log.gz 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27403.log 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34304.log.gz 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35484.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35034.log 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37060.log 2024-04-02T16:13:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.27403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34200.log 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35034.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.27403.log.gz 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37060.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35034.log.gz 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35190.log 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37060.log.gz 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34278.log 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35190.log: /var/log/ceph/ceph-client.admin.34356.log 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34200.log.gz 2024-04-02T16:13:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34278.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35190.log.gz 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35458.log 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34278.log.gz 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33926.log 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35458.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34356.log.gz 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35458.log.gz 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35700.log 2024-04-02T16:13:21.872 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.35700.log: -- /var/log/ceph/ceph-osd.2.log 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33926.log.gz 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36412.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35700.log.gz 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37086.log 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33024.log 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33472.log 2024-04-02T16:13:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33024.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.37086.log.gz --verbose 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.35164.log 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33024.log.gz 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33184.log 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35164.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33472.log.gz 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35726.log 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35164.log.gz 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-04-02T16:13:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35726.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33184.log.gz 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37795.log 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35726.log.gz 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36442.log: gzip/var/log/ceph/ceph-client.admin.37795.log: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37348.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37795.log.gz 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-04-02T16:13:21.875 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36598.log 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37244.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35405.log 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36598.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33708.log 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35405.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36598.log.gz 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33264.log 2024-04-02T16:13:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33708.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35405.log.gz 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36572.log 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33708.log.gz 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34460.log 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33264.log: /var/log/ceph/ceph-client.admin.36572.log: gzip -5 --verbose 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33264.log.gz 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.36009.log 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34460.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36572.log.gz 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34046.log 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34460.log.gz 2024-04-02T16:13:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34382.log 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38059.log 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36009.log.gz 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34382.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38030.log 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34046.log.gz 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34382.log.gz 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33368.log 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38030.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-04-02T16:13:21.878 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35294.log 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%/var/log/ceph/ceph-client.admin.33368.log: -- replaced with /var/log/ceph/ceph-client.admin.38030.log.gz 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33368.log.gz 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27512.log 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35294.log.gz 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33156.log 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37393.log: /var/log/ceph/ceph-client.admin.27512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33521.log 2024-04-02T16:13:21.879 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27512.log.gz 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33156.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33156.log.gz 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34512.log 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33521.log: /var/log/ceph/ceph-client.admin.37509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32972.log 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33521.log.gz 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34982.log 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-04-02T16:13:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35216.log 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.32972.log: /var/log/ceph/ceph-client.admin.34982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34512.log.gz 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32972.log.gz 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34982.log.gz 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37029.log 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35216.log.gz 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33730.log 2024-04-02T16:13:21.881 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37874.log 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37029.log.gz 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35086.log 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37874.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33730.log.gz 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-04-02T16:13:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37874.log.gz 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35647.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.35086.log.gz 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37769.log 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32998.log 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37769.log.gz 2024-04-02T16:13:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.32998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34050.log 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35856.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.32998.log.gz -5 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37419.log 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22640.log 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34050.log.gz 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37926.log 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37419.log.gz 2024-04-02T16:13:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.22640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36494.log 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22640.log.gz 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36087.log 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9%/var/log/ceph/ceph-client.admin.36494.log: -- replaced with /var/log/ceph/ceph-client.admin.37926.log.gz 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37691.log 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36087.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36494.log.gz 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33050.log 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36087.log.gz 2024-04-02T16:13:21.885 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36624.log 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37691.log.gz 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33050.log: /var/log/ceph/ceph-client.admin.36624.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33050.log.gz 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36624.log.gz 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37848.log 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36520.log 2024-04-02T16:13:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37848.log.gz/var/log/ceph/ceph-client.admin.36520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33733.log 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36702.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36520.log.gz 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33420.log 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35008.log 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.admin.33420.log: -- replaced with /var/log/ceph/ceph-client.admin.33733.log.gz 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-04-02T16:13:21.887 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33420.log.gz 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34930.log 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36650.log: /var/log/ceph/ceph-client.admin.35008.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35008.log.gz 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37003.log 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36139.log 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37003.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34930.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.888 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36139.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37003.log.gz 2024-04-02T16:13:21.889 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36139.log.gz 2024-04-02T16:13:21.889 INFO:teuthology.orchestra.run.smithi204.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-02T16:13:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33721.log 2024-04-02T16:13:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36951.log 2024-04-02T16:13:21.913 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-client.admin.33721.log: -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33721.log.gz 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36951.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36951.log.gz 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35674.log 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.admin.35674.log: -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34252.log 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.914 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35674.log.gz 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36243.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33316.log 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34826.log 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33316.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34252.log.gz 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34852.log 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34826.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33316.log.gz 2024-04-02T16:13:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34852.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34826.log.gz 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34852.log.gz 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35752.log 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35963.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35752.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35752.log.gz 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-02T16:13:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37535.log 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33914.log 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37535.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33446.log 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gz 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37535.log.gz 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33914.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33914.log.gz 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34408.log 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36217.log 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34408.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33446.log.gz 2024-04-02T16:13:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27622.log 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34408.log.gz 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36217.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36217.log.gz 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37665.log 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.27622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34434.log 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%/var/log/ceph/ceph-client.admin.37665.log: -- replaced with /var/log/ceph/ceph-client.admin.27622.log.gz 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32926.log 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37665.log.gz 2024-04-02T16:13:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34434.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34434.log.gz 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.32926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33509.log 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%/var/log/ceph/ceph-client.admin.33925.log: -- replaced with /var/log/ceph/ceph-client.admin.32926.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33509.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33509.log.gz 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-04-02T16:13:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34174.log 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38083.log: /var/log/ceph/ceph-client.admin.33212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34174.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34174.log.gz 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33732.log 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37112.log 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36386.log: /var/log/ceph/ceph-client.admin.33732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34022.log 2024-04-02T16:13:21.920 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37112.log: 87.8% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33732.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37112.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33722.log 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34022.log: /var/log/ceph/ceph-client.admin.33722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37613.log 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34022.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33722.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-02T16:13:21.921 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38109.log 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36297.log 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38109.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37613.log.gz 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38109.log.gz 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36297.log: /var/log/ceph/ceph-mds.a.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36297.log.gz 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36269.log 2024-04-02T16:13:21.922 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37190.log 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36165.log: /var/log/ceph/ceph-client.admin.36269.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36269.log.gz 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34226.log 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37190.log: /var/log/ceph/ceph-client.admin.34226.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37190.log.gz 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34226.log.gz 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37587.log 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35353.log 2024-04-02T16:13:21.923 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33924.log: /var/log/ceph/ceph-client.admin.37587.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37587.log.gz 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34567.log 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36895.log 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35353.log: /var/log/ceph/ceph-client.admin.34567.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35353.log.gz 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34567.log.gz 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32896.log 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-04-02T16:13:21.924 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36895.log: /var/log/ceph/ceph-client.admin.32896.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32896.log.gz 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36895.log.gz 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36834.log 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38004.log 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35510.log: /var/log/ceph/ceph-client.0.36834.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36546.log 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38004.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38004.log.gz 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-04-02T16:13:21.925 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36546.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36546.log.gz 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35327.log 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36676.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.36834.log.gz 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33523.log 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35327.log: /var/log/ceph/ceph-client.admin.33523.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35327.log.gz 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33523.log.gz 2024-04-02T16:13:21.926 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33342.log 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34748.log 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36191.log: /var/log/ceph/ceph-client.admin.33342.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33342.log.gz 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36778.log 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37296.log 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34748.log: /var/log/ceph/ceph-client.admin.36778.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34748.log.gz 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36778.log.gz 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36113.log 2024-04-02T16:13:21.927 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37296.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37296.log.gz 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36468.log 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36113.log: /var/log/ceph/ceph-client.admin.36925.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36113.log.gz 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.36468.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36468.log.gz 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35882.log 2024-04-02T16:13:21.928 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37900.log 2024-04-02T16:13:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35595.log: /var/log/ceph/ceph-client.admin.35882.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-04-02T16:13:21.929 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35882.log.gz 2024-04-02T16:13:21.929 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33102.log 2024-04-02T16:13:21.930 INFO:teuthology.orchestra.run.smithi204.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-02T16:13:21.935 INFO:teuthology.orchestra.run.smithi204.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-02T16:13:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37561.log 2024-04-02T16:13:21.939 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37900.log: /var/log/ceph/ceph-client.admin.33102.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37900.log.gz 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33102.log.gz 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37270.log 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33518.log 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37561.log: /var/log/ceph/ceph-client.admin.37270.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37561.log.gz 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37270.log.gz 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33524.log 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33510.log 2024-04-02T16:13:21.940 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33518.log: /var/log/ceph/ceph-client.admin.33524.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33518.log.gz 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33524.log.gz 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz/var/log/ceph/ceph-client.admin.33510.log: 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33510.log.gz 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34800.log 2024-04-02T16:13:21.941 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34774.log: /var/log/ceph/ceph-client.admin.34800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33720.log 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37164.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34800.log.gz 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34018.log 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34695.log 2024-04-02T16:13:21.942 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34018.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37717.log 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33720.log.gz 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34018.log.gz 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.34695.log: 89.7% -- replaced with /var/log/ceph/ceph-client.0.34695.log.gz 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35804.log 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34122.log 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.37717.log: /var/log/ceph/ceph-client.admin.35804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33076.log 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.34122.log: 88.8% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37717.log.gz 2024-04-02T16:13:21.943 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35804.log.gz 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34122.log.gz 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35379.log 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33076.log: /var/log/ceph/ceph-client.admin.33734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33238.log 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35379.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33076.log.gz 88.3% 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34956.log 2024-04-02T16:13:21.944 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35379.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33519.log 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33238.log: /var/log/ceph/ceph-client.admin.34956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35934.log 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.33519.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33238.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34956.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33519.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35908.log 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35934.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35934.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.35908.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35908.log.gz 2024-04-02T16:13:21.945 INFO:teuthology.orchestra.run.smithi040.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-02T16:13:21.948 INFO:teuthology.orchestra.run.smithi204.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-02T16:13:21.949 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-02T16:13:21.951 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-02T16:13:21.962 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-02T16:13:21.970 INFO:teuthology.orchestra.run.smithi040.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-02T16:13:21.983 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-02T16:13:21.984 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-04-02T16:13:21.984 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m0.169s 2024-04-02T16:13:21.985 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m0.755s 2024-04-02T16:13:21.985 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.048s 2024-04-02T16:13:21.988 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-02T16:13:22.022 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-02T16:13:22.023 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-02T16:13:22.023 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m0.225s 2024-04-02T16:13:22.023 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m0.846s 2024-04-02T16:13:22.024 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.116s 2024-04-02T16:13:22.024 INFO:tasks.ceph:Archiving logs... 2024-04-02T16:13:22.025 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi040/log 2024-04-02T16:13:22.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T16:13:22.328 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi204/log 2024-04-02T16:13:22.328 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T16:13:22.413 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T16:13:22.438 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T16:13:22.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T16:13:22.442 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T16:13:22.502 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T16:13:22.512 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T16:13:22.512 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:13:22.514 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T16:13:22.525 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-04-02T16:13:22.525 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-04-02T16:13:22.525 INFO:teuthology.orchestra.run.smithi040.stdout:+hv01.front.sepi 44.4.53.2 2 u 17 64 77 0.086 +1.053 0.814 2024-04-02T16:13:22.525 INFO:teuthology.orchestra.run.smithi040.stdout:*hv02.front.sepi 63.231.80.2 3 u 24 64 77 0.085 -0.543 0.428 2024-04-02T16:13:22.526 INFO:teuthology.orchestra.run.smithi040.stdout:+hv03.front.sepi 74.6.168.72 3 u 24 64 77 0.089 -0.758 0.394 2024-04-02T16:13:22.526 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:13:22.527 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-04-02T16:13:22.527 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-04-02T16:13:22.527 INFO:teuthology.orchestra.run.smithi204.stdout:*hv01.front.sepi 44.4.53.2 2 u 23 64 77 0.068 +0.606 0.746 2024-04-02T16:13:22.527 INFO:teuthology.orchestra.run.smithi204.stdout: hv02.front.sepi 63.231.80.2 3 u 19 64 77 0.090 -0.230 0.640 2024-04-02T16:13:22.527 INFO:teuthology.orchestra.run.smithi204.stdout: hv03.front.sepi 74.6.168.72 3 u 14 64 77 0.136 -1.004 0.646 2024-04-02T16:13:22.528 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-02T16:13:22.528 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T16:13:22.538 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T16:13:22.538 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T16:13:22.547 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T16:13:22.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T16:13:22.565 INFO:teuthology.task.internal:Duration was 784.724231 seconds 2024-04-02T16:13:22.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T16:13:22.576 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T16:13:22.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T16:13:22.578 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T16:13:22.616 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T16:13:22.616 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-02T16:13:22.616 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T16:13:22.668 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-04-02T16:13:22.669 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T16:13:22.679 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T16:13:22.679 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T16:13:22.713 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T16:13:22.753 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T16:13:22.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T16:13:22.780 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T16:13:22.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T16:13:22.865 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T16:13:22.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T16:13:22.880 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T16:13:22.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T16:13:22.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T16:13:22.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T16:13:22.934 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-02T16:13:22.945 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-04-02T16:13:23.154 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T16:13:23.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:13:23.160 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T16:13:23.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T16:13:23.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T16:13:23.211 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T16:13:23.211 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi040 2024-04-02T16:13:23.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T16:13:23.233 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686/remote/smithi204 2024-04-02T16:13:23.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T16:13:23.263 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T16:13:23.263 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T16:13:23.277 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T16:13:23.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T16:13:23.320 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T16:13:23.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T16:13:23.330 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T16:13:23.330 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T16:13:23.333 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T16:13:23.336 INFO:teuthology.orchestra.run.smithi040.stdout: 658642 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 16:13 /home/ubuntu/cephtest 2024-04-02T16:13:23.356 INFO:teuthology.orchestra.run.smithi204.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 16:13 /home/ubuntu/cephtest 2024-04-02T16:13:23.358 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T16:13:23.368 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T16:13:23.497 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T16:13:23.524 DEBUG:teuthology.nuke:shortname: smithi040 2024-04-02T16:13:23.525 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T16:13:23.550 DEBUG:teuthology.nuke:shortname: smithi204 2024-04-02T16:13:23.550 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T16:13:23.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 15:51:04.939783', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRkQdXxPy/y2GsjXPcZmsdMRTTfTw3lb43j9ELZzcr0kLRrXHgO+ltSrf0OT/QX3M+AIKGCfHZzUFk8gB5NL9g='} 2024-04-02T16:13:23.575 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635686', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 15:51:04.941374', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPVksMOgDS4ZluSQ96fhgGWRK7u4t7F6s4evIu+rAK70Q3Q9d12fWJFVIIu/bU7DyogNnL260EZII3oxw/t1W8='} 2024-04-02T16:13:23.611 INFO:teuthology.orchestra.console.smithi040:Power off 2024-04-02T16:13:23.611 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T16:13:23.635 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2024-04-02T16:13:23.636 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:23.652 INFO:teuthology.orchestra.console.smithi204:Power off 2024-04-02T16:13:23.652 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T16:13:23.662 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-02T16:13:23.676 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-04-02T16:13:23.676 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:23.699 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-02T16:13:27.664 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:27.700 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:27.776 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-02T16:13:27.811 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-04-02T16:13:31.778 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:31.812 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T16:13:31.889 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2024-04-02T16:13:31.923 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-04-02T16:13:31.991 INFO:teuthology.orchestra.console.smithi040:Power off completed 2024-04-02T16:13:32.024 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-04-02T16:13:32.129 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} duration: 784.7242312431335 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=26eb7be263674166ba9c2b29a12cbc9c status: fail success: false 2024-04-02T16:13:32.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T16:13:32.211 INFO:teuthology.run:FAIL